Expecting Something for Nothing? The Trials, Tribulations, Successes and Pitfalls of Cross-Cultural Data Collection for an IFTDO/UFHRD Funded Comparative Analysis of HRD Practices

International Journal of HRD Practice, Policy and Research 2016, Vol 1 No 1: 105-108
HRD Viewpoints
Expecting Something for Nothing? The Trials, Tribulations, Successes and Pitfalls of Cross-Cultural Data Collection for an IFTDO/UFHRD Funded Comparative Analysis of HRD Practices
Sophie Mills and Amanda Lee

Download

Is it possible to engage in research requiring the participation of HRD practitioners from a multitude of nations, without offering individual tangible incentives? This viewpoint shares the experiences of our HRD research team in attempting to gather research data from HRD practitioners across the Globe. Issues that potentially indicate tensions between the worlds of HRD practice and academia are reflected upon in the following account. We suggest HRD practitioner awareness of, and connection with, the associated research has a fundamental influence on the relative successes of data collection methods. A review of our experiences of conducting this data collection follows.

Our research was jointly funded by IFTDO (The International Federation for Training and Development Organisations) and UFHRD (The University Forum for HRD)1. We were tasked with conducting a comparative analysis of Human Resource Development (HRD) practices across Africa, Asia and Europe. The guiding research questions addressed in the study involve the scrutiny of the roles, strategies, contribution and driving forces of HRD in each of these three regions.

In order to collect relevant data for this research our methods included accessing the following four main sources:
•Review of relevant literature — incorporating academic and ‘grey’ sources.
•Delphi style panel — set up to determine which countries in Africa, Asia and Europe should be visited to conduct focus groups.
•On-line questionnaire — distributed to IFTDO members and other HRD practitioners across the Globe.
•Country specific focus groups — with countries as identified by the Delphi-style panel.

Literature Review
A thorough review was conducted of the academic and grey literature available to us at the time. The literature review provided the theoretical underpinning for our study and was organized into three distinct sections: the national and international context of each continent; the further and higher education context and finally, the role of HRD practitioners in the respective continents. Thus, it served to inform the questions we produced for our online questionnaire and focus groups.

Delphi-Style Panel
Our plan was to utilise the guidance of experts in order to identify the most appropriate countries we should target for our focus group data collection. We adopted a Delphi-style approach to this process. Delphi panels are designed to access the perspectives of those identified as experts as a means of guidance in relation to the subject under scrutiny. So, within the context of this study, we initially engaged executive members of IFTDO to identify HRD experts likely to be able to assist us with our decision making regarding the most appropriate countries to target from each of the three continents. However, the number of so called experts involved in this process was smaller than would usually be associated with the Delphi approach. We, thus, identified our approach as adopting a ‘Delphi-style’.
As a result of three ‘rounds’ of questioning, the following countries were decided upon: Ireland, to represent Europe; Nigeria, to represent Africa; India and Taiwan, to represent Asia. UK travel restrictions prevented travel to Nigeria, hence the decision to visit two countries in Asia. It is worth noting at this point that Germany was initially suggested as the country to represent Europe. However, after several unsuccessful attempts to organize focus groups it became apparent that in-depth exploration and investigation into the role of HRD and HRD practitioners, was not regarded as a high priority for the German organisations we contacted. As such, we were unable to recruit any volunteers, suggesting a potential tension between academia and HRD practice.

Online Questionnaire
We developed an online questionnaire comprising 26 questions in total, asking participants to share with us details of their experiences of HRD practice within the countries they were based. We utilized the IFTDO membership distribution list to share the online link to our questionnaire via email with potential participants. We also asked HRD specialists in receipt of this email to forward the questionnaire’s link on to any of their contacts involved with HRD. We used a similar approach with other HRD practitioner contact lists at our disposal. We initially provided a deadline for completion of four weeks after the date of the questionnaire’s distribution. We had hoped for approximately 1,000–2,000 completed questionnaires to be submitted to us by this stage.

Unfortunately, a very small number of questionnaires were completed and submitted. This led us to make the decision to reopen the link to the questionnaire and invite via email participation for a second time. At this stage we provided potential participants with a few additional weeks in which to complete the questionnaire. We reemphasized the importance of participation in this exciting international research project and hoped for better results after the second deadline. We were again very disappointed with the small number (under 30 completed questionnaires in total) of questionnaires submitted after the second deadline had passed.
We were surprized and puzzled by the lack of participation in our research. Whilst we appreciated and understood that questionnaire completion, especially for online questionnaires, was notorious for achieving low completion levels nonetheless we could not work out why we had received fewer than 30 completed questionnaires from across the Globe. Had we been somewhat naïve in our assumption that HRD practitioner members of IFTDO and our other HRD practitioner networks would share a vested interest in contributing to the outcome of this important and potentially impactful international study?

We revisited the structure and content of our questionnaire at this point in order to identify any possible reasons for the low response rate. We agreed that the questionnaire’s 26 questions were a reasonable number to expect participants to answer within approximately half an hour. We decided to redistribute the questionnaire for a final time and to express our gratitude in advance for the time and effort participants would be providing us with. Unfortunately, our efforts did not result in any increase in participation.

This significant lack of quantitative data available for our research meant that we needed to rethink our approach to analysis. We decided to continue with the impending focus groups and modify our research methods so as to utilise the completed questionnaire data in the development of our focus groups. However, it was clear that we would not be able to make any generalisations from our questionnaire data, as we had originally hoped we would be able to do.

Focus Groups
Two focus groups were conducted in Ireland, by Professor Stewart. Two focus groups were facilitated by Sophie Mills in India. One focus group was conducted by Sophie Mills in Taiwan. One focus group was facilitated by Sophie Mills and took place with Nigerian HRD practitioners via Skype (due to travel restrictions in place at that time).

The support and hospitality we received from each country was very generous and much appreciated. The qualitative data received from focus groups provided us with a very important insight into perceptions of and attitudes toward HRD practice within each of these countries. It was also more noticeable than ever from these focus groups that generalisations across continents within this context would be very difficult, if not impossible, to achieve. However, the data we were able to glean from this part of data collection enabled us to understand better the relationships that were at play between each country’s government, education systems and industries. This encouraged us to consider future opportunities for widening our research to involve the participation of HRD practitioners from many more countries, and also to investigate further differences experienced between sectors.

On Reflection …
Our experiences of data collection were varied. The Delphi-style study proved to be a very useful means of identifying the countries upon which we would conduct the ‘deep-dive’ element of our research. The high points of data collection were definitely related to our focus group involvement. We remain indebted to the HRD practitioners involved in putting together and attending these focus groups. They were generous with their time and efforts and we certainly benefitted hugely from their participation.

A significant low point, and perhaps the biggest learning point for us to consider in the extension of this research, has to be associated with lack of HRD practitioner engagement in the completion of our online questionnaire. To what extent did this demonstrate tension between the perspectives of HRD theorists and those of HRD practitioners? Was this perhaps evidence of a dichotomy in existence between efforts associated with academic HRD theory development in contrast/conflict with HRD practice? Even though HRD practitioners within this context claimed generally to be supportive of advancing their understandings of practices associated with employee learning and development, their interest and, thus, engagement appeared to be somewhat lacking. It seems that without some form of personal commitment in this type of study, HRD practitioners are unlikely to ‘waste their time’ answering a questionnaire.

We have certainly learned that it isn’t enough to send an email and cross our fingers in the hope that HRD practitioners across the world, a) have the time to devote to supporting this type of international research, and b) are inclined to contribute in order to influence our final report. It is felt that in future development of this research we arguably need to consider the possibility of incentivizing participation by maybe offering gift vouchers, raffle prizes or other tangible rewards, to those that agree to complete the questionnaire.

Perhaps it is even worth us contemplating modifying completely the research methods adopted for future development of this research project. A suggestion was made, for example, by an audience member at the 2015 IFTDO World Conference when our initial results were being presented by Sophie Mills. The audience member suggested that we could engage more with representatives from each country not only in the distribution of questionnaires, but also in their development. This, it was posited, would address any potential cultural or language barriers prior to the questionnaire’s distribution. Also, it would be likely to enable us to access far more potential participants, rather than relying on one main source of distribution. Similarly, when Amanda Lee presented a progress update at the 2015 International UFHRD conference, feedback from delegates suggested offering incentives to questionnaire participants could be used to facilitate completion.

Finally, we would like to express our sincere gratitude to those HRD practitioners who did take part in our research. We considered the focus group participation to be a resounding success. We have also been able to effectively utilise the secondary data gleaned from our review of relevant academic and grey literature. A full report of our research findings is due to be released via the IFTDO network shortly.

Our next plan is to gain additional funding to extend our study further to include additional countries, taking with us the lessons we have learned along the way. It is likely that future HRD practitioner engagement in our study will be determined by the level of interaction we can gain in the development of our research.

Note
1. The research team comprised Professor Jim Stewart (Professor of HRD at Coventry University) Amanda Lee (Senior Lecturer in HRM at Coventry University) and Sophie Mills (Senior Lecturer at Coventry University).

The Authors
Amanda Lee is Senior Lecturer in human resource management at Coventry University, with over 12 years’ experience in academia and 15 years’ human resource practitioner experience gained within the National Health Service, construction and retail sectors.
Sophie Mills is Senior Lecturer in human resource management at Coventry University, with over 12 years’ experience in academia and 8 years’ human resource practitioner experience gained within UK Local Government and the security industry.

Journal 1 Volume 1