Re-examining Digital-Learning Readiness in Higher Education: Positioning Digital Competencies as Key Factors and a Profile Application as a Readiness Tool

Reexamining Digital-Learning Readiness in Higher Education: Positioning Digital Competencies as Key Factors and a Profile Application as a Readiness Tool

## TODD BLAYONE

_EILAB, University of Ontario Institute of Technology, Canada_ [email protected]

Readiness for digital learning is an international research domain exploring the preparedness of people and contexts for successful technology-rich education. This study, focused on higher education, reviews the readiness literature and positions digital competencies as factors within it. A key limitation identified is that despite the position of digital knowledge, skills and attitudes as highly significant factors, readiness instruments are limited by inconsistent conceptualization and unidimensional operationalization of digital competencies. In response, the General Technology Competency and Use (GTCU) framework and accompanying online Digital Competency Profiler (DCP) application, developed in Canada and used over a decade in several national contexts, are reviewed as an alternative readiness-assessment apparatus. Findings from a pilot observational study comparing: (a) self-reported digital competencies (using DCP indicators), and (b) the quality of authentic digital-learning performances, are highlighted in support. Through continued application development and international collaboration, the researchers are pursuing contextually-sensitive and non-exclusionary readiness-assessment tools that help individuals and groups better prepare for successful digital learning.

INTRODUCTION

Digital learning, an umbrella term inclusive of online, distance and blended learning (Siemens, Gašević, & Dawson, 2015), and incorporating mobile learning (Alhassan, 2016; Crompton, Burke, Gregory, & Gräbe, 2016), has profoundly influenced higher education on a global scale. There are more than 40 million higher-education students in the world taking one or more of their classes online (Atkins, 2013). Of course, the practices of digital learning are diverse, shaped by values, epistemologies and learning models in specific institutional and socio-cultural contexts (Aparicio, Bacao, & Oliveira, 2016).

Some forms of digital learning, such as MOOCs (Massive Open Online Courses) focus on making educational content, often from professors at Western universities, globally accessible (De Corte, Engwall, & Teichler, 2016). Some focus on building educational management systems that balance individualized and cooperative forms of learning, and enhance institutional sustainability (Dalsgaard & Paulsen, 2009; Paulsen, 2003, 2008). Others promote transactional learning in which collaboration functions to reduce confirmation bias and construct knowledge (Garrison, 2016). Still others seek to correct power imbalances in traditional education, and facilitate the development of online communities to address authentic problems using open resources (Blayone, vanOostveen, Barber, DiGiuseppe, & Childs, 2017; vanOostveen, Davidson, Naffi, Price, & Blayone, 2016; vanOostveen, DiGiuseppe, Barber, Blayone, & Childs, 2016). Taken together as a general milieu, digital learning achieves greatest success—which may be defined in relation to program sustainability, participant satisfaction, or other learning outcomes (Bates, 2015)—when the socio-cultural environment, supporting digital infrastructure, host institutions, and human participants are well prepared, or _ready_ .

The research literature addressing readiness for digital learning is international, with researchers seeking to align local systems of higher education with an increasingly global and digital society. This study organizes the literature, positions digital competencies as readiness factors within it, and offers a bridge between readiness research and specialized digital-competency research with a view to improving readiness assessment at the participant level. One significant problem identified is that despite the position of digital competencies as highly significant factors, readiness instruments are limited by inconsistent and unidimensional indicators that have not yet incorporated digital-competency frameworks. In response, the General Technology Competency and Use (GTCU) framework, authored by Desjardins

(Desjardins, Lacasse, & Belair, 2001), and the accompanying online Digital Competency Profiler (DCP) application (Desjardins, Davidson, Blayone, vanOostveen, & Childs, 2015), are positioned as an alternative, readinessassessment apparatus. By reviewing the GTCU from conceptual and operational perspectives, and highlighting findings from a pilot, performanceanalysis study (Blayone, Mykhailenko, vanOostveen, & Barber, 2018), it is argued that the DCP offers researchers a useful option for probing the readiness of students and teachers for digital learning in higher education.

Methodology

A critical review (Grant & Booth, 2009) was conducted of the digitallearning readiness literature to establish its conceptual structure and position digital competencies as readiness factors. Items were selected according to three criteria. First, owing to a cross-cultural research perspective, a broadly international representation of peer-reviewed articles was sought. Second, studies were included to: (a) represent the conceptual scope of the research, and (b) position digital-competency factors in readiness frameworks and survey instruments. Finally, effort was made to organize the literature thematically, stopping when saturation was achieved (Mason, 2010). In the end, 76 studies were analysed, with 25 dated between 2015 and 2017. This review process was supplemented by a parallel review of digital-competency frameworks and measurement methodologies (p.11), as part of ongoing framework development (Desjardins et al., 2015) and investigation of digital competencies as online-learning readiness factors in Ukraine (Blayone, Mykhailenko, Kokhan, et al., 2018; Blayone et al., 2017).

The guiding research questions were:

1. What is the conceptual structure of digital-learning readiness research?

2. What is the general position of digital competencies as factors in digital-learning-readiness frameworks and instruments?

3. Does the DCP, based on the General Technology Competency and Use framework, address limitations in current readiness instruments?

FINDINGS

Conceptual Structure of Readiness for Digital Learning

## _**Origins**_

Readiness for digital learning has roots in at least two research traditions: learning psychology, and technology-systems development. Within learning psychology, Thorndike (1932) introduced “readiness” to account for observed differences in the intensity of human responses to stimuli which would be expected, all things being equal, to generate the same intensity of response. Thorndike developed this idea into a “law” of learning, interpreted by Knowles (1973) as “the circumstances under which a learner tends to be satisfied or annoyed, to welcome or to reject” an educational intervention (p. 20). This law became a staple of behaviourist educational psychology at the level of individual learners (Allen, 2007). Several decades later, engineers of the National Aeronautics and Space Administration (NASA) introduced the concept of “technology readiness levels” to determine the maturity of technology systems, and the economic and human requirements for effective use (Sullivan, 1970).

By the 1990s, models and instruments were introduced to assess the readiness of individuals, groups and societies for effective use of digital technologies (Sopu, Chisaki, & Usagawa, 2016). For example, Warner, Christie & Choi (1998) explored the readiness of adults for distance vocational training (Hung, Chou, & Chen, 2010). With special reference to computer-conferencing environments in higher education, Gunawardena & Duphorne (2001) explored learner readiness, extending the work of Eastmond (1994).

## _**Terminology**_

More recently, readiness for digital learning has been addressed through a cluster of related terms including: e-readiness (Ilgaz & Gülbahar, 2015), online readiness (Farid, 2014), e-learning/e-learner readiness (Leigh & Watkins, 2005; Mosa, Naz’ri bin Mahrin, & Ibrrahim, 2016; Rohayani, Kurniabudi, & Sharipuddin, 2015), online-learning readiness (Horzum, Kaymak, & Gungoren, 2015), mobile-learning readiness (Lin, Lin, Yeh, Wang, & Jansen, 2015), and e-learning acceptance (Teo, 2010). This tight family of constructs is well represented in English-language scholarship around the world. It is noteworthy that a few researchers prefer “pre-

paredness” to readiness as a way to emphasize those behaviours that can be learned, rather than those linked to personality (Parkes, Stein, & Reading, 2015).

## _**Definitions**_

At the highest level of analysis, readiness for digital learning refers to the measure of the degree to which a nation, country or economy is prepared to obtain benefits from digital educational technologies (Sopu et al., 2016). At a medium level of analysis, organizational definitions view readiness as an assessment of factors to be considered if institutions are to be successful or experience benefits with an e-learning strategy (Farid, 2014; Ilgaz & Gülbahar, 2015; Odunaike, Olugbara, & Ojo, 2013); or, the capacity of an institution to integrate ICTs into their practices, and manage digital divides (Chipembele, Chipembele, Bwalya, & Bwalya, 2016). This last definition, authored in Zambia, introduces a socio-economic aspect, highlighting inequalities of technology access. Yet, it should be emphasized that access is a necessary but insufficient condition for readiness because it does not address _effective_ use (Eastin & LaRose, 2000; A.J.A.M. van Deursen & van Dijk, 2011a, 2014).

At the participant level, an early definition relates readiness to three facets: (a) student preferences for form of delivery, (b) student confidence in electronic communication for learning, and (c) a learner’s ability to engage in autonomous learning (Warner et al., 1998). More recent definitions highlight attitudes and abilities essential to successful online learning for both educators and students (Gunawardena & Duphorne, 2001; Schrum & Hong, 2002); those qualities of learners that impact satisfaction with online learning, and can be distinguished from environmental variables (Pillay, Irving, & Tones, 2007); the level of technology adoption by individuals for learning (Hussin, Manap, Amir, & Krish, 2012; Lin et al., 2015); and finally, the extent to which learners are prepared to apply their digital experience during preparation, delivery and completion phases (Gay, 2016).

These definitions emphasize different actors (e.g., users, students and educators), desirable attitudes (e.g., optimism and acceptance), learning contexts (e.g., formal and informal) and outcomes (e.g., learning satisfaction and “significant benefits”). Yet, as depicted in Figure 1, they present a _common structure_ consisting of current stakeholder/contextual characteristics in relation to a future digital-learning implementation _and_ some idea of success.

**Figure 1.** Common Structure of Readiness-for-Digital-Learning Definitions.

## _**Research purposes**_

The purposes (or problems) undergirding readiness research are many and multifaceted. Some researchers are most concerned with general adoption of digital technologies as an avenue of educational reform and innovation (Peng, Tsai, & Wu, 2006). Others, who function in contexts of high digital adoption, worry about the sustainability of exiting digital-learning programs and the related issue of student completion rates (van Rooij & Zirkle, 2016). Still others seek to anticipate the cognitive and emotional challenges facing students who are transitioning from face-to-face environments (Schrum & Hong, 2002). Similarly, some pursue readiness research to increase learner satisfaction and improve the quality of digital-learning (Ilgaz & Gülbahar, 2015).

## _**Organizing themes**_

Three dominant themes were constructed to organize the research. First, the literature can be clustered based on _level of analysis_ (McGill, Klobas, & Renzi, 2014; van Rooij & Zirkle, 2016). Macro-level analyses address factors at the organizational, regional or national levels (Beetham & Sharpe, 2007; Bui, Sankaran, & Sebastian, 2003). Addressing readiness at the national level is of particular interest in contexts where digital learn-

ing is in its infancy, and factors such as national Internet penetration, mobile subscribers, institutional culture and technological infrastructure are key (Aung & Khaing, 2016; Hrtoćová, Kohout, Rohlíková, & Zounek, 2015; Sopu et al., 2016). Micro-level readiness research turns its attention to human stakeholders with both students (Dray, Lowenthal, Miszkiewicz, Ruizć Primo, & Marczynski, 2011; Parkes et al., 2015) and teachers (Gay, 2016; Hung, 2016) garnering significant attention. In some cases, a more specific demographic, such as adult learners, is selected (Ilgaz & Gülbahar, 2015).

A second organizing theme relates to the _twin dimensionality_ of readiness. Researchers have conceptualized several readiness factors either (a) deductively, through the development of readiness frameworks (Demir & Yurdugül, 2015; Mosa et al., 2016) or (b) inductively, through groundedtheoretical analysis (Al-Araibi, Mahrin, & Mohd, 2016; Rohayani et al., 2015), document analyses (Schrum & Hong, 2002), or by surveying those involved in both successful and unsuccessful e-learning initiatives (McGill et al., 2014). These factors cluster into two groups: _technology_ perceptions and abilities, on the one hand, and _learning_ dispositions and abilities, on the other hand.

A third organizing theme, relates to _level of focus_ . The bulk of the readiness literature addresses digital learning as a general “milieu” to be distinguished from traditional, in-class education. However, more specialized research is appearing, related to specific: (a) technologies, such as mobile (Hussin et al., 2012; Lin et al., 2015; Shorfuzzaman & Alhussein, 2016; Yeap, Ramayah, & Soto-Acosta, 2016); (b) forms of learning, such as collaborative-constructivist (van Rooij & Zirkle, 2016); or (c) socio-economic and cultural factors in particular national contexts (Alaaraj & Ibrahim, 2014; Chipembele et al., 2016).

Positioning Digital Competencies as Readiness Factors

A thorough analysis of digital competence and closely related constructs (e.g., digital literacy, digital skills, and e-competence) was performed by Ala-Mutka (2011) as a contribution to the EU’s DIGCOMP project (Ferrari, 2012, 2013; Janssen & Stoyanov, 2012; Pérez-Escoda & FernándezVillavicencio, 2016; Vuorikari, Punie, Gomez, & Van Den Brande, 2016). In order to explore the position of digital competencies as factors in digitallearning readiness research, we adopt her compact definition of digital competencies as _sets of knowledge, skills and attitudes relating to the purposeful and effective use of digital technologies_ (Ala-Mutka, 2011).

Digital competencies as factors within readiness frameworks

Two recent literature reviews seek to identify the most significant factors in readiness frameworks. Within a Malaysian context, Al-Araibi et al. (2016) offer a thematic analysis of 17 framework studies dated from 2004 to 2013. ICT-related skills and attitudes emerge as the most significant factors—mentioned in 15 studies. A systematic review by Turkish scholars includes 25 frameworks divided into three groups, addressing the readiness of: (a) learners (12 models), (b) teachers (7 models), and (c) institutions (10 models) (Demir & Yurdugül, 2015). For each group, the constituent factors are organized, and their distribution calculated. Technology competency is found to be the strongest factor in relation to both teacher readiness (included in 100% of the models) and learner readiness (included in 83% of the models). Technology-competency factors also appear in 50% of the institutional-readiness models.

Together, these studies highlight digital competencies as a prominent complex of factors within readiness frameworks. Chipembele (2016) concurs, arguing that the technology skills and confidence levels of students and teachers are more significant than organizational factors for assessing readiness, suggesting that where levels are low, organizational investments in digital-learning infrastructure are likely to be underutilized.

Digital competency items within readiness instruments

Since at least 2000, researchers in diverse higher education spaces have developed, validated and applied digital-learning readiness instruments. To position digital competencies as readiness factors, ten validated instruments were selected for comparative analysis. These included six instruments listed by Wladis & Samuels (2016), an influential early instrument (Parasuraman, 2000), another developed in Turkey (Aydın & Tasci, 2005), and two specialized instruments addressing mobile-learning readiness (Lin et al., 2015) and teacher-specific readiness (Hung, 2016). Key comparative data were extracted (Table 1, pages 434-436) to establish: (a) the strength and breadth of constituent knowledge, skills and attitudes addressed by each instrument; and (b) the manner in which these constructs are operationalized as self-report items.

Across the ten instruments reviewed, the highest percentage of digitalcompetency items is 100% (Parasuraman, 2000; Watkins, Leigh, & Triner, 2004), and the lowest is 31% (Kerr, Rynearson, & Kerr, 2006). The average concentration of digital-competency-related items is 64%. This finding

is consistent with the strength of digital competency as a conceptual dimension in readiness frameworks, described above. Remaining items relate mainly to self-direction, self-regulation or desire for social interaction—all individual constructs playing a prominent role in digital-learning research (Garrison, 2011, 2016; Garrison & Archer, 2000).

Among the digital competency sub-factors, all selected instruments, except one (Kerr et al., 2006), address _attitudes_ , with four instruments focusing exclusively on such factors (Hung, 2016; Hung et al., 2010; Lin et al., 2015; Parasuraman, 2000). Items related to self-efficacy or confidence of technology use predominates. Five instruments include a focus on _knowledge_ and _skills_ , with one utilizing direct (“I am capable of…”) ability indicators (Bernard, Brauer, Abrami, & Surkes, 2004). Other instruments present unsystematic sets of beliefs (“I think I would be able to…”) (Watkins et al., 2004); knowledge and experience questions (“Do you know…” or “How often do you…”) (Dray et al., 2011); or, items related to technology access (Aydın & Tasci, 2005; Watkins et al., 2004). With respect to the inclusion of attitudinal items, an outlier is the Test of Online Learning Success (TOLS) (Kerr et al., 2006). All the items in this instrument (representing direct, firstperson, claims) are constructed to reflect a skill rather than an attitude. The authors defend this choice by noting that, in social-psychology research, one typically finds weak correlations between attitudes and behaviours (Kerr et al., 2006). Overall, aside from a general preference for attitudinal items, there is little consistency between these instruments in how digital competencies are conceptualized and measured.

|**Indicator Type(s)**|Attitudes<br>Factors measured with second-<br>person statements as indicators<br>of four key attitudes.|Attitudes and Skills<br>First-person, direct, self-<br>assessment of procedural skills<br>and attitudes. Indicators relate<br>to basic technical and social<br>interactions.|Attitudes and Skills<br>Mix of frst-person claims<br>of current skills and beliefs<br>about future activities and<br>applications. Items relate<br>to a mix of basic technical,<br>informational and social<br>interactions.| |---|---|---|---| |**Sample Digital Competency Items**|All 36 items (100%)<br>You like the idea of doing business on computers because you<br>aren’t limited to business hours.<br>People ask your advice on new technologies.<br>Sometimes you think that technology systems are not<br>designed for use by ordinary people.|19 items of 37 (51%)<br>I can easily access the Internet…for my studies.<br>I possess suffcient computer keyboarding skills for doing<br>work online.<br>I am comfortable communicating electronically....|All 27 items (100%)<br>I have access to a computer and Internet.<br>I think I would be able to take notes on a computer...or<br>communicate effectively online.<br>I think I would be able to complete my work even when there<br>are online distractions.<br>I feel that prior experiences with online technologies…are<br>important to my success.| |**Factors**|1. Technology optimism<br>2. Technology innovativeness<br>3. Technology discomfort<br>4. Technology insecurity|1. Confdence in digital skills<br>2. Beliefs re: online learning<br>3. Self-direction and initiative<br>4. Desire for interaction|1. Technology access<br>2. Online skills<br>3. Online activity motivation<br>4. Online video/audio<br>5. Internet discussions<br>6. Importance to my success| |**Research**|Parasuraman,<br>A. (2000),<br>USA|Bernard,<br>R. M., et<br>al. (2004),<br>Canada|Watkins, R.,<br>et al. (2004),<br>USA|

**----- Start of picture text -----**<br> Indicator Type(s)<br>Attitudes and Skills Indirect, assessment of employees’ basic access, computer skills and attitudes (related to general acceptance). Skills First-person, direct, self- assessment of basic technical, informational and social skills. Attitudes and Skills First-person, direct, self- assessment of current technical skills, computer-related confidence and likes covering a range of technical, informational, computational and social interactions.<br>**----- End of picture text -----**<br>

||**Indicator Type(s)**|Attitudes and Skills<br>Indirect, assessment of<br>employees’ basic access,<br>computer skills and attitudes<br>(related to general acceptance).|Skills<br>First-person, direct, self-<br>assessment of basic technical,<br>informational and social skills.|Attitudes and Skills<br>First-person, direct, self-<br>assessment of current<br>technical skills, computer-<br>related confdence and likes<br>covering a range of technical,<br>informational, computational<br>and social interactions.| |---|---|---|---|---| |||||| |Comparison of Digital Competency Factors in Readiness Instrume|**Sample Digital Competency Items**|11 items of 30 (37%)<br>Do you think your employees are able to access the<br>Internet at work?<br>Do your employees possess basic computer skills (such<br>as keyboarding, using a mouse, creating, saving and<br>editing fles)?|14 items of 45 (31%)<br>I am capable of learning new technologies.<br>I can download new software when necessary.<br>I need online courses because of my geographical<br>distance from universities.|17 items of 20 (85%)<br>I know how to install software to support my learning<br>using computers.<br>I feel confdent…accessing the Internet.<br>I like using computers for research.<br>“I enjoy working on tasks on a computer that I can do by<br>following directions.| ||**Factors**|1. People characteristics<br>2. Employees’ self-development<br>3. Technology access & skills<br>4. Level of innovation|1. Computer Skills<br>2. Independent Learning<br>3. Dependent Learning<br>4. Need for Online Learning|1. Technical skills<br>2. Computer self-effcacy<br>3. Learning preferences<br>4. Attitudes to computers| ||**Research**|Aydın, C. H. and<br>D. Tasci (2005),<br>Turkey|Kerr, M. S., et<br>al. (2006), USA|Pillay, H., et al.<br>(2007), Australia|

**----- Start of picture text -----**<br> Indicator Type(s)<br>Attitudes First-person, direct, self-assessment of confidence related to technical, informational and social interactions. Knowledge and Skills Questions on technology access, frequency of use, knowledge, and device preferences. Attitudes First-person, direct, self-assessment of confidence in mobile and social interactions. Attitudes First-personal, direct, self-assessment of confidence related to social interactions.<br>**----- End of picture text -----**<br>

|**Indicator Type(s)**|Attitudes<br>First-person, direct, self-<br>assessment of confdence related<br>to technical, informational and<br>social interactions.|Knowledge and Skills<br>Questions on technology access,<br>frequency of use, knowledge, and<br>device preferences.|Attitudes<br>First-person, direct, self-<br>assessment of confdence in<br>mobile and social interactions.|Attitudes<br>First-personal, direct, self-<br>assessment of confdence related<br>to social interactions.| |---|---|---|---|---| |||||| |**Sample Digital Competency Items**|11 items of 18 (61%)<br>I feel confdent using Microsoft Offce programs.<br>I have motivation to learn (online).<br>I like to share my ideas with others (online).<br>I feel confdent using online communication tools.|18 items of 32 (56%)<br>Do you know how to install software on your computer<br>for course purposes?<br>Do you know how to open, view and/or print...?|14 items of 19 (74%)<br>I feel confdent in using mobile systems…<br>I like studying via mobile learning systems because I<br>am able to study anytime.|8 items of 18 (44%)<br>I feel confdent in posting questions online.<br>I am distracted…when learning online….<br>I am confdent about applying learning from my online<br>course to my teaching job.| |**Factors**|1. Computer self-effcacy<br>2. Self-directed learning<br>3. Learner control<br>4. Motivation for e-learning<br>5. Online comm. self-effcacy|1. Learner characteristics<br>2. Technology capabilities|1. m-Learning self-effcacy<br>2. Mobile optimism<br>3. Self-directed Learning|1. Comm. self-effcacy<br>2. Institutional support<br>3. Self-directed e-learning<br>4. Learning self-effcacy| |**Research**|Hung, M.-L.,<br>et al. (2010),<br>Taiwan|Dray, B. J., et al.<br>(2011), USA|Lin, H.-H., et al.<br>(2015), Taiwan|Hung, M.-L.<br>(2016), Taiwan|

DISCUSSION

Gaps in current readiness research

There appears to be conceptual and operational gaps between readiness researchers, on the one hand, and digital-competency researchers, on the other hand. This can be illustrated by comparing the reference lists of readiness articles—for example (Dray et al., 2011; Hung, 2016; Lin et al., 2015)—to a major literature review of digital-competencies research (Litt, 2013). Common references include the work of Bandura (1997) on self-efficacy, and a study using self-efficacy as a digital-competency measure (Eastin & LaRose, 2000). A large body of research, published over the past two decades, focused on (a) developing multi-dimensional frameworks, and (b) exploring measurement methodologies, are strikingly absent. For example, the following list of key digital-competency research goes unmentioned:

- The diverse, digital-skills measurement and methodological work of Hargittai and her research team, formally centred in America, and now in Zurich, Switzerland (Hargittai, 2002, 2004, 2009; Hargittai & Hsieh, 2012; Hargittai & Shafer, 2006).

- The “digital thinking skills” framework and performance observation work of Eshet in Israel (Eshet, 2002, 2004, 2012; Eshet-Alkalai & Amichai-Hamburger, 2004; Eshet-Alkalai & Chajut, 2009, 2010), which includes a notable longitudinal study (Eshet-Alkalai & Chajut, 2009, 2010).

- The digital-skills framework, performance observation and instrument-development work of van Deursen and van Dijk in the Netherlands (A.J.A.M van Deursen, Courtois, & van Dijk, 2014; A.J.A.M. van Deursen, Helsper, & Eynon, 2014; A.J.A.M. van Deursen & van Dijk, 2010, 2011a, 2011b; van Dijk & van Deursen, 2014).

- The digital-abilities conceptualization and measurement work of Helsper in the UK (Helsper & Eynon, 2013), recently joined by van Deursen (Helsper & van Deursen, 2015; Helsper, van Deursen, & Eynon, 2015).

- The digital-competency assessment models and related studies of Cartelli and Calvani in Italy (Calvani, Cartelli, Fini, & Ranieri, 2008; Calvani, Fini, Ranieri, & Picci, 2012; Cartelli, 2010; Cartelli, Latium, & Cassino, 2014), which although not offering a single,

consolidated conceptual framework, have been richly generative, particularly through Cartelli’s editorship of the _International Journal of Digital Literacy and Digital Competence_ (http://www. igi-global.com/journal/international-journal-digital-literacydigital/1170).

- The EU’s digital-competency project (Ala-Mutka, 2011; AlaMutka, Punie, & Redecker, 2008; Ferrari, 2012; Janssen & Stoyanov, 2012), which published its DIGCOMP 1.0 framework specification in 2013 (Ferrari, 2013), and is undergoing revision (Pérez-Escoda & Fernández-Villavicencio, 2016; Vuorikari et al., 2016).

- The digital-competency framework and instrument development work of Desjardins and the EILAB research group in Canada (Desjardins, 2005; Desjardins et al., 2001; Desjardins & Peters, 2007; Desjardins & vanOostveen, 2015), conducted in a higher educational context of social-constructivist, fully online learning.

Within this research, three items stand out as particularly relevant. First, effective use of digital technologies involves several types of activity, including technical or operational, informational or content-related, and communicational or social—three dimensions shared by most frameworks. Second, although measuring both attitudes and behaviours are important, self-reported confidence and skills are sometimes unreliable predictors of performance. This is why observational research is recommended as vital part of a full, digital-competency-measurement program (Bradlow, Hoch, & Hutchinson, 2002; Hargittai & Shafer, 2006; Litt, 2013; A.J.A.M. van Deursen, Helsper, & Eynon, 2015). Finally, the socio-political environments of researchers shape the manner in which digital competencies are conceptualized and measured.

A Representative Digital Competencies Model

The General Technology Competency and Use (GTCU) framework, authored by Desjardins (2005; 2015; 2001; 2007) in Canada, has several strengths. First, it addresses multiple, formally-conceptualized, dimensions of digital activity. Second, it is coupled with a proprietary, Digital Competency Profiler, described below. Third, over the past decade, it has been used successfully in higher education to measure the digital competencies of students and teachers for fully online learning (Desjardins & vanOostveen, 2015; Desjardins, vanOostveen, Bullock, DiGiuseppe, & Robertson, 2010;

DiGiuseppe, Partosoedarso, vanOostveen, & Desjardins, 2013). Fourth, it is directly informed by performance observations conducted using the superb, digital capture and analysis resources of the EILAB (Blayone, Mykhailenko, vanOostveen, & Barber, 2018). Finally, it benefits through a program of cross-cultural collaboration with research partners in North America, and Western and Eastern Europe (Blayone, Mykhailenko, Kokhan, et al., 2018; Blayone et al., 2017).

Conceptual dimensions of the GTCU

The GTCU is constructed upon the IEEE’s definition of computer hardware: “physical equipment used to process, store, or transmit computer programs or data” (IEEE, 1990). In this way, as shown in Figure 2, three foundational orders of interaction and use are theorized: epistemological (process), informational (store) and social (transmit). Successful users of digital technology develop knowledge, skills and attitudes (KSAs) in each of these dimensions, aligned with the context of use and desired outcome(s). A technical dimension, addressing operational skills, is added as a prerequisite for optimal functioning in the other dimensions.

**Figure 2.** Orders of the GTCU Framework.

Epistemological competencies relate to computational and data-processing activities. Although this once required programming skills, today, sophisticated procedures are supported by software applications, including spreadsheets, concept-mapping tools, photo-manipulation programs, and personal planners. The key ingredients for epistemological competence are

deep conceptual knowledge in a domain, and the ability to translate this knowledge into operational procedures using the functions provided by a computer application. Primary benefits relate to effective problem solving, hypothesis development, and cognitive partnering (Jonassen, 1995).

Informational competencies relate to interactions between humans and digital knowledge artefacts. The most relevant KSAs fall into four groups. The first focuses on finding relevant information. This requires conceptual and procedural knowledge, and skills related to using search tools and building queries. A second group relates to assessing relevance, usefulness, reliability and validity. A third group involves the organization, comparison and synthesis of selected information with other data. Finally, in order to transform information into knowledge artefacts, abilities related to production, revision and publication of information are required.

Social competencies address facets of digital communication, collaboration and sharing. They include conceptual knowledge of appropriate social interaction, and procedural knowledge for successful communication using a variety of applications (e.g., email, social media and video conferencing). Social-media applications, in particular, also require an ability to maintain appropriate online identities and assess privacy risks. The abilities to match tools with desired outcomes, and promote respect and trust are also vital.

The Digital Competency Profiler

The GTCU was initially operationalized using a self-report instrument designed to study the digital abilities of Canadian teachers (Desjardins, 2005; Desjardins et al., 2001). Over time, it was used to profile professors and students at a Canadian technology university (Desjardins & vanOostveen, 2015; Desjardins et al., 2010; DiGiuseppe et al., 2013). Freely available to research partners, the DCP, which incorporated this instrument, is a developing online platform. The current version includes rich support for data collection and aggregation, and profile generation and comparison. Development plans include enhancing the core features and extending personalization, internationalization and data-analysis functionality (Roland van Oostveen, EILAB Director, personal communication, June 9, 2017).

Data collection and aggregation

The DCP collects data with socio-demographic questions and 26 activity items using two measures: _frequency_ and _confidence_ of activity, conceptualized as twin indicators of digital competence. Frequency of activity addresses transferable procedural knowledge reinforced through ongoing use

of digital devices. Confidence, directly aligned with perceived self-efficacy (Bandura, 1993), relates to an individual’s willingness to tackle complex problems positively, and extend abilities already acquired. Both frequency and confidence are measured in relation to a preferred device (e.g., smartphone, tablet, laptop, etc.). By using twin indicators and associating activities with specific types of devices, the DCP collects a depth of self-report data well beyond most readiness instruments. Moreover, as a persistent international database housing ever-increasing amounts of profile data, researchers gain the ability to compare data across contexts and over time.

Profile generation and comparison

The DCP calculates an individual’s Relative Competency Index score and generates profiles presented in the form of a colour-coded, aster-plot visualization (Figure 3). In order to establish a general interpretive context, one can perform profile comparisons with selected groups. For research partners with access to the analysis module, a variety of group profiles can be generated and compared using data selection filters, including gender, age, education level, location and organization. Another feature, under development, allows an individual or group to maintain multiple profiles to compare over time. A raw dataset can be extracted for advanced analysis in external applications such as Excel or SPSS.

**Figure 3.** Digital Competency Profiler Visualization.

The DCP and performance observations

In 2017, a pilot study was completed to develop a performance-analysis methodology, and compare an individual’s self-reported digital competencies (using DCP indicators) to their activity while addressing an authentic, digital-learning scenario (Blayone, Mykhailenko, vanOostveen, & Barber, 2018). Each scenario was designed with six constituent tasks mapped to self-report indicators from three dimensions of interaction. Tablets were used as activity devices owing to the increasing role of mobile technology in education, and the call from researchers to address the device gap in performance analysis (Litt, 2013; A.J.A.M. van Deursen et al., 2015). A videoanalysis application (Noldus The Observer XT 13) was used for reviewing multi-perspectival video, and coding: (a) task completion strategy, (b) observed levels of comfort and engagement, (c) performance problems (e.g., navigation and touch input issues), (d) situational inhibitors such as frustration and fatigue, and (e) scenario-completion times.

Findings showed a general pattern of positive correlations between selfreports and performance quality at the task and aggregate levels. Based on the full, mixed-methods data set, participants at the high and low ends were positioned on opposite ends of a hypothetical readiness spectrum. Through analysis of the middle participants, a threshold GTCU score was established as a tentative marker below which students “at risk” might be located. Through analysis of the performance video, it became apparent that several participants below this threshold displayed significant difficulties completing their assigned scenario. In the end, the DTP application showed an ability to predict the strongest and weakest performers, and help identify an “at risk” segment, confirming its potential as a viable readiness instrument.

CONCLUSION

This study described the conceptual structure of readiness for digitallearning research, including origins, definitions, guiding purposes and organising themes. Moreover, it demonstrated that digital competencies represent highly significant factors in readiness frameworks and self-report instruments. However, it presented evidence that readiness researchers tend to rely upon unidimensional conceptualizations of digital competency and inconsistent sets of indicators that depart from the frameworks and findings of digital-competency researchers.

To address this gap, the GTCU framework’s DCP application was positioned as a promising digital-learning readiness tool. This proprietary, online application combines both behavioural and attitudinal measures, relates competency to a variety of device-specific activities, and offers rich profile visualizations and data comparison functionality. Moreover, an initial performance study highlighted the DCP’s potential as a reliable predictor of digital-learning performance.

The DCP is being adopted by educational institutions internationally to measure digital competencies of teachers and students. At the same time, the EILAB team continues to enhance the application’s core features, increase localization support, and build support for surveys addressing other dimensions of readiness (e.g., cultural predispositions and learning preferences). Researchers are invited to participate in these efforts through associations with the EILAB (eilab.ca). This program of development and global collaboration is particularly important when readiness assessment is conceptualized as an inclusionary endeavour—as an opportunity to design contextually-responsive learning experiences, facilitate guided self-assessment and personal-development plans, and ultimately, help more students and teachers around the world experience _successful_ digital learning.

References

- Al-Araibi, A. A. M., Mahrin, M., & Mohd, R. C. (2016). A systematic literature review of technological factors for e-learning readiness in higher education. _Journal of Theoretical and Applied Information Technology, 93_ (2), 500-521.

- Ala-Mutka, K. (2011). _Mapping digital competence: Towards a conceptual understanding_ . Seville, Spain: Institute for Prospective Technological Studies (IPTS), European Commission, Joint Research Centre. Retrieved from http://ftp.jrc.es/EURdoc/JRC67075_TN.pdf.

- Ala-Mutka, K., Punie, Y., & Redecker, C. (2008). _Digital competence for lifelong learning_ . Seville, Spain: Institute for Prospective Technological Studies (IPTS), European Commission, Joint Research Centre. Retrieved from http://ftp.jrc.es/EURdoc/JRC48708.TN.pdf.

- Alaaraj, H., & Ibrahim, F. W. (2014). An overview and classification of e-readiness assessment models. _International Journal of Scientific and Research Publications, 4_ (12), 1-5.

- Alhassan, R. (2016). Mobile learning as a method of ubiquitous learning: Students’ attitudes, readiness, and possible barriers to implementation in higher education. _Journal of Education and Learning, 5_ (1), 176. doi:10.5539/jel. v5n1p176

- Allen, S. J. (2007). Adult learning theory & leadership development. _Leadership Review, 7_ , 26-37.

- Aparicio, M., Bacao, F., & Oliveira, T. (2016). An e-learning theoretical framework. _Journal of Educational Technology & Society, 19_ (1), 292-307. doi:www.jstor.org/stable/jeductechsoci.19.1.292

- Aung, T. N., & Khaing, S. S. (2016). Challenges of implementing e-Learning in developing countries: A review. In T. T. Zin (Ed.), _Genetic and Evolutionary Computing: Advances in Intelligent Systems and Computing_ (pp. 405411). Switzerland: Springer International Publishing.

- Aydın, C. H., & Tasci, D. (2005). Measuring readiness for e-learning: reflections from an emerging country. _Educational Technology and Society, 8_ (4), 244257. doi:www.jstor.org/stable/jeductechsoci.8.4.244

- Bandura, A. (1993). Perceived self-efficacy in cognitive development and functioning. _Educational Psychologist, 28_ (2), 117-148. doi:10.1207/ s15326985ep2802_3

- Bandura, A. (1997). _Self-efficacy: The exercise of control_ . New York: W. H. Freeman.

- Bates, T. (2015). _Teaching in a digital age: Guidelines for designing teaching an learning_ . Vancouver, BC. Open Access Electronic Book retrieved from https://opentextbc.ca/teachinginadigitalage/

- Beetham, H., & Sharpe, R. (2007). _Rethinking pedagogy for a digital age: Designing for 21st century learning_ . New York, NY: Routledge.

- Bernard, R. M., Brauer, A., Abrami, P. C., & Surkes, M. (2004). The development of a questionnaire for predicting online learning achievement. _Distance Education, 25_ (1), 31-47. doi:10.1080/0158791042000212440

- Blayone, T., Mykhailenko, O., Kokhan, M., Kavtaradze, M., vanOostveen, R., & Barber, W. (2018). Profiling the digital readiness of higher education students for transformative online learning in the post-soviet nations of Georgia and Ukraine. Manuscript accepted for publication. International Journal of Educational Technology in Higher Education. doi:10.1186/s41239-0180119-9

- Blayone, T., Mykhailenko, O., vanOostveen, R., & Barber, W. (2018). Ready for digital learning? A mixed-methods exploration of surveyed technology competencies and authentic performance activity. _Education and Information Technologies, 23_ (3), 1377-1402. doi:10.1007/s10639-017-9662-6

- Blayone, T., Mykhailenko, O., VanOostveen, R., Grebeshkov, O., Hrebeshkova, O., & Vostryakov, O. (2017). Surveying digital competencies of university students and professors in Ukraine for fully online collaborative learning. _Technology, Pedagogy and Education,_ 1-18. doi:10.1080/147593 9X.2017.1391871

- Blayone, T., vanOostveen, R., Barber, W., DiGiuseppe, M., & Childs, E. (2017). Democratizing digital learning: Theorizing the Fully Online Learning Community model. _International Journal of Educational Technology in Higher Education, 14_ (13), 1-16. doi:10.1186/s41239-017-0051-4

- Bradlow, E. T., Hoch, S. J., & Hutchinson, J. W. (2002). An assessment of basic computer proficiency among active internet users: Test construction, calibration, antecedents and consequences. _Journal of Educational and Behavioral Statistics, 27_ (3), 237-253. doi:10.3102/10769986027003237

- Bui, T. X., Sankaran, S., & Sebastian, I. M. (2003). A framework for measuring national e-readiness. _International Journal of Electronic Business, 1_ (1), 3-22. doi:10.1504/ijeb.2003.002162

Calvani, A., Cartelli, A., Fini, A., & Ranieri, M. (2008). Models and instruments for assessing digital competence at school. _Journal of e-Learning and Knowledge Society-English Version, 4_ (3), 183-193.

- Calvani, A., Fini, A., Ranieri, M., & Picci, P. (2012). Are young generations in secondary school digitally competent? A study on Italian teenagers. _Computers & Education, 58_ (2), 797-807. doi:10.1016/j.compedu.2011.10.004

- Cartelli, A. (2010). _Frameworks for digital competence assessment: Proposals, instruments, and evaluation_ . Paper presented at the Informing Science & IT Education Conference (INSITE), Cassino, Italy.

- Cartelli, A., Latium, S., & Cassino, I. C. G. (2014). Special issue on digital literacy and digital competence: Facts problems, needs and trends. _International Journal of Digital Literacy and Digital Competence, 5_ (4), iv-x.

- Chipembele, M., Chipembele, M., Bwalya, K. J., & Bwalya, K. J. (2016). Assessing e-readiness of the Copperbelt University, Zambia: Case study. _The International Journal of Information and Learning Technology, 33_ (5), 315332. doi:10.1108/IJILT-01-2016-0005

- Crompton, H., Burke, D., Gregory, K. H., & Gräbe, C. (2016). The Use of Mobile Learning in Science: A Systematic Review. _Journal of Science Education and Technology_ , 1-12. doi:10.1007/s10956-015-9597-x

- Dalsgaard, C., & Paulsen, M. F. (2009). Transparency in cooperative online education. _The International Review of Research in Open and Distributed Learning, 10_ (3), 1-13. doi:10.19173/irrodl.v10i3.671

- De Corte, E., Engwall, L., & Teichler, U. (2016). The hype of MOOCs. In E. De Corte, L. Engwall, & U. Teichler (Eds.), _From Books to MOOCS? Emerging Models of Learning and teaching in Higher Education_ (Vol. 88, pp. xvxxv). London, UK: Portland Press.

- Demir, Ö., & Yurdugül, H. (2015). The exploration of models regarding e-learning readiness: Reference model suggestions. _International Journal of Progressive Education, 11_ (1), 173-194.

- Desjardins, F. J. (2005). Information and communication technology in education: A competency profile of francophone secondary school teachers in Ontario. _Canadian Journal of Learning and Technology/La revue canadienne de l’apprentissage et de la technologie, 31_ (1), 1-14. doi:10.21432/ T2PG69

- Desjardins, F. J., Davidson, A.-L., Blayone, T., vanOostveen, R., & Childs, E. (2015). _General Technology Competency and Use: Foundations_ . Information Poster. University Ontario Institute of Technology. Oshawa, Canada. Retrieved from http://eilab.ca/wp-content/uploads/2016/01/GTCU-Poster-V22000px.jpg

- Desjardins, F. J., Lacasse, R., & Belair, L. M. (2001). _Toward a definition of four orders of competency for the use of information and communication technology (ICT) in education_ . Paper presented at the Computers and Advanced Technology in Education, Banff, Canada. http://eilab.ca/wp-content/ uploads/2013/04/2001CATE.pdf

- Desjardins, F. J., & Peters, M. (2007). Single-course approach versus a program approach to develop technological Competencies in pre-service language teaching. In M.-A. Kassen, L. Lavine, K. Murphy-Judy, & M. Peters (Eds.), _Preparing and Developing Technology Proficient L2 Teachers_ (pp. 3-21). Texas, USA: Texas State University.

- Desjardins, F. J., & vanOostveen, R. (2015). Faculty and student use of digital technology in a “laptop” university. In S. Carliner, C. Fulford, & N. Ostashewski (Eds.), _EdMedia: World Conference on Educational Media and Technology 2015_ (pp. 990-996). Montreal, Canada: Association for the Advancement of Computing in Education (AACE).

- Desjardins, F. J., vanOostveen, R., Bullock, S., DiGiuseppe, M., & Robertson, L. (2010). Exploring graduate student’s use of computer-based technologies for online learning. In J. Herrington & C. Montgomerie (Eds.), _EdMedia: World Conference on Educational Media and Technology 2010_ (pp. 440-444): Association for the Advancement of Computing in Education (AACE).

- DiGiuseppe, M., Partosoedarso, E., vanOostveen, R., & Desjardins, F. J. (2013). Exploring competency development with mobile devices. In M. B. Nunes & M. McPherson (Eds.), _International Association for Development of the Information Society (IADIS) International Conference on e-Learning_ (pp. 384-388). Prague, Czech Republic: International Association for Development of the Information Society.

- Dray, B. J., Lowenthal, P. R., Miszkiewicz, M. J., RuizćPrimo, M. A., & Marczynski, K. (2011). Developing an instrument to assess student readiness for online learning: A validation study. _Distance Education, 32_ (1), 29-47. doi:1 0.1080/01587919.2011.565496

- Eastin, M. S., & LaRose, R. (2000). Internet self-efficacy and the psychology of the digital divide. _Journal of Computer Mediated Communication, 6_ (1), 0-0. doi:10.1111/j.1083-6101.2000.tb00110.x

- Eastmond, D. V. (1994). Adult distance study through computer conferencing. _Distance Education, 15_ (1), 128-152. doi:10.1080/0158791940150109

- Eshet, Y. (2002). _Digital literacy: A new terminology framework and its application to the design of meaningful technology-based learning environments_ . Paper presented at the ED-MEDIA 2002 World Conference on Educational Multimedia, Hypermedia & Telecomunications, Denver, Colorado.

- Eshet, Y. (2004). Digital literacy: A conceptual framework for survival skills in the digital era. _Journal of Educational Multimedia and Hypermedia, 13_ (1), 93-106.

- Eshet, Y. (2012). Thinking in the digital era: A revised model for digital literacy. _Issues in Informing Science & Information Technology, 9_ , 267. doi:10.1089=cpb.2008.0264

- Eshet-Alkalai, Y., & Amichai-Hamburger, Y. (2004). Experiments in digital literacy. _CyberPsychology & Behavior, 7_ (4), 421-429.

- Eshet-Alkalai, Y., & Chajut, E. (2009). Changes over time in digital literacy. _CyberPsychology & Behavior, 12_ (6), 713-715. doi:10.1089=cpb.2008.0264

Eshet-Alkalai, Y., & Chajut, E. (2010). You can teach old dogs new tricks: The factors that affect changes over time in digital literacy. _Journal of Information Technology Education, 9_ , 173-181.

- Farid, A. (2014). Student online readiness assessment tools: A systematic review approach. _Electronic Journal of e-Learning, 12_ (4), 375-382.

- Ferrari, A. (2012). _Digital competence in practice: An analysis of frameworks_ . Seville, Spain: Institute for Prospective Technological Studies (IPTS), European Commission, Joint Research Centre. Retrieved from http://ftp.jrc.es/ EURdoc/JRC68116.pdf.

- Ferrari, A. (2013). _DIGCOMP: A framework for developing and understanding digital competence in Europe_ . Seville, Spain: Institute for Prospective Technological Studies (IPTS), European Commission, Joint Research Centre Retrieved from http://publications.jrc.ec.europa.eu/repository/handle/ JRC83167.

- Garrison, D. R. (2011). _E-learning in the 21st century: A framework for research and practice_ (2nd ed.). New York: Taylor & Francis.

- Garrison, D. R. (2016). _Thinking collaboratively: Learning in a community of inquiry_ . New York: Routledge.

- Garrison, D. R., & Archer, W. (2000). _A transactional perspective on teaching and learning: A framework for adult and higher education_ . New York: Pergamon.

- Gay, G. (2016). An assessment of online instructor e-learning readiness before, during, and after course delivery. _Journal of Computing in Higher Education, 28_ (2), 199-220. doi:10.1007/s12528-016-9115-z

- Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. _Health Information & Libraries Journal, 26_ (2), 91-108. doi:10.1111/j.1471-1842.2009.00848.x

- Gunawardena, C. N., & Duphorne, P. L. (2001). _Which learner readiness factors, online features, and CMC related learning approaches are associated with learner satisfaction in computer conferences?_ Paper presented at the Annual Meeting of the American Educational Research Association, Seattle, Washington.

- Hargittai, E. (2002). Beyond logs and surveys: Inćdepth measures of people’s web use skills. _Journal of the American Society for Information Science and Technology, 53_ (14), 1239-1244. doi:10.1002/asi.10166

- Hargittai, E. (2004). Classifying and coding online actions. _Social Science Computer Review, 22_ (2), 210-227. doi:10.1177/0894439303262560

- Hargittai, E. (2009). An update on survey measures of web-oriented digital literacy. _Social Science Computer Review, 27_ (1), 130-137. doi:10.1177/0894439308318213

- Hargittai, E., & Hsieh, Y. P. (2012). Succinct survey measures of webuse skills. _Social Science Computer Review, 30_ (1), 95-107. doi:10.1177/0894439310397146

- Hargittai, E., & Shafer, S. (2006). Differences in actual and perceived online skills: the role of gender. _Social Science Quarterly, 87_ (2), 432-448. doi:10.1111/j.1540-6237.2006.00389.x

Helsper, E. J., & Eynon, R. (2013). Distinct skill pathways to digital engagement. _European Journal of Communication, 28_ (6), 696-713. doi:10.1177/0267323113499113

- Helsper, E. J., & van Deursen, A. J. A. M. (2015). Digital skills in Europe: Research and policy. _Digital Divides: The New Challenges and Opportunities of e-Inclusion, 195_ , 125.

- Helsper, E. J., van Deursen, A. J. A. M., & Eynon, R. (2015). _Tangible Outcomes of Internet Use_ . Digital Skills to Tangible Outcomes project report. Retrieved from http://www.oii.ox.ac.uk/research/projects/?id=112

- Horzum, M. B., Kaymak, Z. D., & Gungoren, O. C. (2015). Structural equation modeling towards online learning readiness, academic motivations, and perceived learning. _Educational Sciences: Theory and Practice, 15_ (3), 759770. doi:10.12738/estp.2015.3.2410

- Hrtoćová, N., Kohout, J., Rohlíková, L., & Zounek, J. (2015). Factors influencing acceptance of e-learning by teachers in the Czech Republic. _Computers in Human Behavior, 51_ , 873-879. doi:10.1016/j.chb.2014.11.018

- Hung, M.-L. (2016). Teacher readiness for online learning: Scale development and teacher perceptions. _Computers & Education, 94_ , 120-133. doi:10.1016/j.compedu.2015.11.012

- Hung, M.-L., Chou, C., & Chen, C.-H. (2010). Learner readiness for online learning: Scale development and student perceptions. _Computers & Education, 55_ (3), 1080-1090. doi:10.1016/j.compedu.2010.05.004

- Hussin, S., Manap, M. R., Amir, Z., & Krish, P. (2012). Mobile learning readiness among Malaysian students at higher learning institutes. _Asian Social Science, 8_ (12), 276. doi:10.5539/ass.v8n12p276

- IEEE. (1990). IEEE standard computer dictionary: A compilation of IEEE standard computer glossaries (pp. 218). New York, NY: The Institute of Electrical and Electronics Engineers.

- Ilgaz, H., & Gülbahar, Y. (2015). A snapshot of online learners: e-Readiness, e-Satisfaction and expectations. _The International Review of Research in Open and Distributed Learning, 16_ (2), 1-12. doi:10.19173/irrodl. v16i2.2117

- Janssen, J., & Stoyanov, S. (2012). _Online Consultation on Experts’ Views on Digital Competence_ . Seville, Spain: Institute for Prospective Technological Studies (IPTS), European Commission, Joint Research Centre.

- Jonassen, D. H. (1995). Computers as cognitive tools: Learning with technology, not from technology. _Journal of Computing in Higher Education, 6_ (2), 4073. doi:10.1007/bf02941038

- Kerr, M. S., Rynearson, K., & Kerr, M. C. (2006). Student characteristics for online learning success. _The Internet and Higher Education, 9_ (2), 91-105. doi:10.1016/j.iheduc.2006.03.002

- Knowles, M. (1973). _The adult learner: A neglected species_ . Houston, Texas: Gulf Pub, Co.

- Leigh, D., & Watkins, R. (2005). E-learner success: Validating a self-assessment of learner readiness for online training. In _ASTD 2005 Research-to-Practice Conference Proceedings_ (pp. 121-131). Alexandria, VA, USA: ATD.

- Lin, H.-H., Lin, S., Yeh, C.-H., Wang, Y.-S., & Jansen, J. (2015). Measuring mobile learning readiness: Scale development and validation. _Internet Research, 26_ (1), 265-287. doi:10.1108/IntR-10-2014-0241

- Litt, E. (2013). Measuring users’ internet skills: A review of past assessments and a look toward the future. _New Media & Society, 15_ (4), 612-630. doi:10.1177/1461444813475424

- Mason, M. (2010). Sample size and saturation in PhD studies using qualitative interviews. _Forum qualitative Sozialforschung/Forum: Qualitative Social Research, 11_ (3, Art. 8), 63 paragraphs. doi:nbn-resolving.de/ urn:nbn:de:0114-fqs100387

- McGill, T. J., Klobas, J. E., & Renzi, S. (2014). Critical success factors for the continuation of e-learning initiatives. _The Internet and Higher Education, 22_ , 24-36. doi:10.1016/j.iheduc.2014.04.001

- Mosa, A. A., Naz’ri bin Mahrin, M., & Ibrrahim, R. (2016). Technological aspects of e-learning readiness in higher education: A review of the literature. _Computer and Information Science, 9_ (1), 113-127. doi:10.5539/cis. v9n1p113

- Odunaike, S. A., Olugbara, O. O., & Ojo, S. O. (2013). E-learning implementation critical success factors. In _Proceedings of International MultiConference of Engineers and Computer Scientists 2013_ (Vol. 1, pp. 560-565). Hong Kong: International Association of Engineers (IAENG).

- Parasuraman, A. (2000). Technology Readiness Index (TRI) a multiple-item scale to measure readiness to embrace new technologies. _Journal of Service Research, 2_ (4), 307-320. doi:10.1177/109467050024001

- Parkes, M., Stein, S., & Reading, C. (2015). Student preparedness for university e-learning environments. _The Internet and Higher Education, 25_ , 1-10. doi:10.1016/j.iheduc.2014.10.002

- Paulsen, M. F. (2003). _Online education and learning management systems: Global e-learning in a Scandinavian perspective_ (1st ed.). Bekkestua, Norway: NKI Forlaget.

- Paulsen, M. F. (2008). Cooperative online education. _Seminar.net: International Journal of Media, Technology and Lifelong Learning, 4_ (2), 1-20.

- Peng, H., Tsai, C. C., & Wu, Y. T. (2006). University students’ selfćefficacy and their attitudes toward the Internet: the role of students’ perceptions of the Internet. _Educational Studies, 32_ (1), 73-86. doi:10.1080/03055690500416025

- Pérez-Escoda, A., & Fernández-Villavicencio, N. G. (2016). Digital competence in use: from DigComp 1 to DigComp 2. In _Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality_ (pp. 619-624). Salamanca, Spain: ACM.

- Pillay, H., Irving, K., & Tones, M. (2007). Validation of the diagnostic tool for assessing tertiary students’ readiness for online learning. _High Education Research & Development, 26_ (2), 217-234. doi:10.1080/07294360701310821

- Rohayani, A. H., Kurniabudi, & Sharipuddin. (2015). A literature review: Readiness factors to measuring e-learning readiness in higher education. _Procedia Computer Science, 59_ , 230-234. doi:10.1016/j.procs.2015.07.564

- Schrum, L., & Hong, S. (2002). From the field: Characteristics of successful tertiary online students and strategies of experienced online educators. _Education and Information Technologies, 7_ (1), 5-16. doi:10.1023/a:1015354423055

- Shorfuzzaman, M., & Alhussein, M. (2016). Modeling Learners’ Readiness to Adopt Mobile Learning: A Perspective from a GCC Higher Education Institution. _Mobile Information Systems, 2016_ , 1-10. doi:10.1155/2016/6982824

- Siemens, G., Gašević, D., & Dawson, S. (2015). _Preparing for the digital university: A review of the history and current state of distance, blended, and online learning_ . Retrieved from http://linkresearchlab.org/PreparingDigitalUniversity.pdf

- Sopu, H. T., Chisaki, Y., & Usagawa, T. (2016). Use of Facebook by secondary school students at Nuku’alofa as an indicator of e-readiness for e-learning in the Kingdom of Tonga. _The International Review of Research in Open and Distributed Learning, 17_ (4), 203-223. doi:10.19173/irrodl.v17i4.2333

- Sullivan, E. V. (1970). The issue of readiness in the design and organization of the curriculum: A historical perspective. In R. W. Burns & G. D. Brooks (Eds.), _Curriculum Design in a Changing Society_ (Vol. 10, pp. 39-48). Englewood Cliffs, New Jersey: Educational Technology Publications.

- Teo, T. (2010). Development and validation of the E-learning Acceptance Measure (ElAM). _The Internet and Higher Education, 13_ (3), 148-152. doi:10.1016/j.iheduc.2010.02.001

- Thorndike, E. L. (1932). _The fundamentals of learning_ . New York, NY: Teachers College Bureau of Publications.

- van Deursen, A. J. A. M., Courtois, C., & van Dijk, J. A. G. M. (2014). Internet skills, sources of support, and benefiting from Internet use. _International Journal of Human-Computer Interaction, 30_ (4), 278-290. doi:10.1080/104 47318.2013.858458

- van Deursen, A. J. A. M., Helsper, E. J., & Eynon, R. (2014). _Measuring digital skills_ . Digital Skills to Tangible Outcomes Project Report. Retrieved from http://www.oii.ox.ac.uk/research/projects/?id=112

- van Deursen, A. J. A. M., Helsper, E. J., & Eynon, R. (2015). Development and validation of the Internet Skills Scale (ISS). _Information, Communication & Society_ , 1-20. doi:10.1080/1369118X.2015.1078834

- van Deursen, A. J. A. M., & van Dijk, J. A. G. M. (2010). Measuring internet skills. _International Journal of Human-Computer Interaction, 26_ (10), 891916. doi:10.1080/10447318.2010.496338

- van Deursen, A. J. A. M., & van Dijk, J. A. G. M. (2011a). Internet skills and the digital divide. _New Media & Society, 13_ (6), 893-911. doi:10.1177/1461444810386774

- van Deursen, A. J. A. M., & van Dijk, J. A. G. M. (2011b). Internet skills performance tests: Are people ready for eHealth? _Journal of Medical Internet Research, 13_ (2). doi:10.2196/jmir.1581

- van Deursen, A. J. A. M., & van Dijk, J. A. G. M. (2014). The digital divide shifts to differences in usage. _New Media & Society, 16_ (3), 507-526. doi:10.1177/1461444813487959

- van Dijk, J. A. G. M., & van Deursen, A. J. A. M. (2014). _Digital skills: Unlocking the information society_ . New York: Palgrave Macmillan.

- van Rooij, S. W., & Zirkle, K. (2016). Balancing pedagogy, student readiness and accessibility: A case study in collaborative online course development. _The Internet and Higher Education, 28_ , 1-7. doi:10.1016/j.iheduc.2015.08.001

- vanOostveen, R., Davidson, A.-L., Naffi, N., Price, D., & Blayone, T. (2016). _Exploring Problem-Based Learning in higher education: Case studies about the changing roles of the teacher and the learner_ . Paper presented at the Canadian Society for the Study of Education Annual Conference 2016, Calgary, Alberta, Canada.

- vanOostveen, R., DiGiuseppe, M., Barber, W., Blayone, T., & Childs, E. (2016). New conceptions for digital technology sandboxes: Developing a Fully Online Learning Communities (FOLC) model. In G. Veletsianos (Ed.), _EdMedia 2016: World Conference on Educational Media and Technology_ (pp. 665-673). Vancouver, B.C., Canada: Association for the Advancement of Computing in Education (AACE).

- Vuorikari, R., Punie, Y., Gomez, S. C., & Van Den Brande, G. (2016). _DigComp 2.0: The Digital Competence Framework for Citizens. Update Phase 1: The Conceptual Reference Model_ . Retrieved from Luxembourg Publication Office of the European Union. EUR 27948 EN.: http://publications.jrc. ec.europa.eu/repository/handle/JRC101254

- Warner, D., Christie, G., & Choy, S. (1998). The readiness of VET clients for flexible delivery including on-line learning. _Brisbane: Australian National Training Authority_ .

- Watkins, R., Leigh, D., & Triner, D. (2004). Assessing readiness for e- learning. _Performance Improvement Quarterly, 17_ (4), 66-79. doi:10.1111/j.1937-8327.2004.tb00321.x

- Wladis, C., & Samuels, J. (2016). Do online readiness surveys do what they claim? Validity, reliability, and subsequent student enrollment decisions. _Computers & Education, 98_ , 39-56. doi:10.1016/j.compedu.2016.03.001

- Yeap, J. A., Ramayah, T., & Soto-Acosta, P. (2016). Factors propelling the adoption of m-learning among students in higher education. _Electronic Markets_ , 1-16. doi:10.1007/s12525-015-0214-x