Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

e-ISSN: 2737-632x

Pgs 19-36

Received: June, 2023

Approved: December, 2023

DOI:

https://doi.org/10.31876/ie.

v8i1.263

http://www.revista-

iberoamericana.org/index.

php/es

How to cite:

Maridueña, M., Núñez, P.,

Bejarano, L. (2024) Learning

analytics as tools for academic

monitoring of virtual students

of University Technological

Institutes. Revista

Iberoamericana De

educación, 8 (1)

* PhD., Instituto Tecnológico Superior

“Espíritu Santo”, Universidad de

Guayaquil milton.mariduenaa@ug.edu.ec,

mrmariduena@tes.edu.ec,

https://orcid.org/0000-0002-8876-1896

* MSc. Instituto Tecnológico Superior

“Espíritu Santo”, pnunez@tes.edu.ec,

https://orcid.org/0000-0003-3870-8517

* MSc., Universidad de Guayaquil,

luz.bejaranoo@ug.edu.ec,

https://orcid.org/0000-0002-8494-0571

Learning analytics as a tool for academic

monitoring of virtual students of

University Technological Institutes

Las analíticas de aprendizaje como herramientas de monitoreo académico

de estudiantes virtuales de Institutos Tecnológicos Universitarios

Learning analytics como ferramenta de acompanhamento académico de

alunos virtuais em Institutos Tecnológicos Universitários

Milton Rafael Maridueña Arroyave*

Patricia Alexandra Núñez Panta**

Luz Marina Bejarano Ospina***

Abstract

This article presents four categories of learning analytics tools:

dashboards, specific tools, ad hoc tools, and learning analytics

frameworks. It also describes the features of several tools within each

of these categories: (1) Moodle Dashboard and Moodle's default

reporting tool; (2) the Interactions tool and the Teamwork

Assessment Tool; (3) SNAPP, GraphFES, and Moodle Engagement

Analytics; and (4) VeLA and GISMO. The study focuses on how

these tools can be used to analyze courses by collecting actual data

from a course that extensively used forums, wikis, web resources,

videos, quizzes, and assignments. The subsequent discussion

highlights how these different tools complement each other and

suggests the incorporation of basic dashboards into learning

platforms and the adoption of external frameworks for learning

analytics.

Keywords: Learning analytics - User interaction - Tools - Learning

Analytics

Resumen

Este artículo presenta cuatro categorías de herramientas de análisis

del aprendizaje: paneles de control, herramientas específicas,

herramientas ad hoc y marcos de análisis del aprendizaje. Además,

describe las características de varias herramientas dentro de cada una

de estas categorías: (1) Moodle Dashboard y la herramienta de

informes predeterminada de Moodle; (2) la herramienta de

Interacciones y la Herramienta de Evaluación del Trabajo en Equipo;

(3) SNAPP, GraphFES y Moodle Engagement Analytics; y (4) VeLA

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

y GISMO. El estudio se centra en cómo estas herramientas pueden

utilizarse para analizar cursos mediante la recopilación de datos

reales de un curso que utilizó extensamente foros, wikis, recursos

web, videos, cuestionarios y tareas. La discusión posterior destaca

cómo estas diferentes herramientas se complementan entre sí y

sugiere la incorporación de paneles básicos en las plataformas de

aprendizaje y la adopción de marcos externos para el análisis del

aprendizaje.

Palabras clave: Analíticas de aprendizaje – Interación de usuarios -

Herrramientas

Resumo

Este artigo apresenta quatro categorias de ferramentas de análise da

aprendizagem: painéis de controlo, ferramentas específicas,

ferramentas ad hoc e quadros de análise da aprendizagem. Descreve

as características de várias ferramentas dentro de cada uma destas

categorias: (1) o Painel de Controlo do Moodle e a ferramenta de

relatório predefinida do Moodle; (2) a ferramenta Interacções e a

Ferramenta de Avaliação do Trabalho em Equipa; (3) SNAPP,

GraphFES e Moodle Engagement Analytics; e (4) VeLA e GISMO.

O estudo centra-se na forma como estas ferramentas podem ser

utilizadas para analisar cursos, recolhendo dados reais de um curso

que utilizou extensivamente fóruns, wikis, recursos Web, vídeos,

questionários e trabalhos. A discussão subsequente destaca a forma

como estas diferentes ferramentas se complementam e sugere a

incorporação de painéis básicos nas plataformas de aprendizagem e

a adoção de quadros externos para a análise da aprendizagem.

Palavras-chave: Análise da aprendizagem - Interação dos

utilizadores - Ferramentas - Análise da aprendizagem

INTRODUCTION

The integration of Information and Communication Technologies

(ICT) in educational processes offers a novel way of educating in

both face-to-face and distance learning environments. An

outstanding example of this aspect is the use of platforms for virtual

learning environments, such as Learning Management Systems

(LMS). These LMSs provide support for both fully online and

blended learning. In online and blended learning, due to the absence

of direct physical interaction, instructors and course coordinators

require tools that allow them to track student progress. LMSs collect

large amounts of information about student interactions, but this

information is often stored in LMS databases in the form of raw data,

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

making the extraction and processing of relevant information often a

complex process (Macfadyen & Dawson, 2012)..

The records stored in learning management systems (LMS) contain

a large amount of data related to interactions between students and

teachers, as well as access to resources, virtual activities and

functions of the system itself. This data can provide information on

how and when students complete their assignments, participate in the

course, among other aspects. However, extracting meaningful data

and converting this information into actionable knowledge represents

a challenge. New educational disciplines, such as educational data

mining, academic analytics or learning analytics, offer diverse but

converging approaches, methodologies, techniques and tools aimed

at simplifying this transformation process.

Educational data mining encompasses a variety of techniques that

focus on obtaining educational data through the use of statistical

machine learning algorithms and data mining, with the purpose of

conducting analysis and addressing research questions in the

educational field (Romero & Ventura, 2010) . On the other hand,

academic analytics is approached differently, emphasizing the

analysis of institutional data related to students, and, therefore, it is

more oriented toward decision making related to institutional policies

(Goldstein & Katz, 2005); (Goldstein P. , 2005). Finally, the central

goal of learning analytics is to "measure, collect, analyze and

generate reports on data about students and their contexts, in order to

understand and optimize the learning process and the environments

in which it takes place." (Fergusson, 2012).

From the above it is clear that, despite certain distinctions among the

three disciplines, they all share the common goal of understanding

teaching and learning for the purpose of making informed

educational decisions aimed at improving the learning process

(Agudo-Peregrina & Iglesias-Pradas, 2014)..

Today, a wide variety of tools are available that simplify educational

data collection and analysis for the purpose of learning analytics. A

general way to categorize these tools would be as follows.

(Hernandez-Garcia & Conde, 2014).:

• Dashboards, both those of general use that are compatible

with various platforms and those specific to each platform,

are intended to provide visual and summarized information

about the activity on the platform by various actors in the

learning process, mainly students and teachers.

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

• Ad hoc tools, designed and developed on a customized basis,

are aimed at monitoring and analyzing highly specific

information, adapted to particular situations and contexts.

• Learning Analytics tools focus on the analysis of specific

aspects and seek to provide detailed information, often with

specialized representations. They are often versatile and work

on multiple platforms.

• Learning analytics frameworks and tools aim to standardize

learning ontologies and implement them in various systems.

In addition, they seek to explore learner behaviors in different

educational contexts and provide customizable visual

representations of information to the user.

Given the wide range of applications of learning analytics, the main

purpose of this research is to describe some of them and to apply

them by comparing their results using a common data set of courses

delivered in a learning management system (LMS) such as Moodle.

This comparison will highlight the usefulness, advantages and

disadvantages of the various approaches and perspectives, as well as

how they can complement each other. Therefore, the study is divided

into two well-defined parts: first, the various learning analytics tools

that will be analyzed will be presented, and then the empirical part

will address the results, including a comparison of the tools following

the analysis of data sets from previously taught courses. Finally, the

study will conclude by drawing conclusions about the results

obtained by applying the various tools.

Analysis of learning analytics tools

This research covers both cross-platform tools and LMS (Learning

Management System) specific tools. Different versions of Moodle

are required to test the different tools, as not all analytical tools are

available for all versions of Moodle.

Most of these tools analyze user interaction from the LMS log data.

This means that most tools extract and transform data from the

mdl_log database table. Until version 2.7, each developer could

potentially add their logs to this table from an application, leading to

log formats that might not be "standard".

This problem was solved by defining a new logging system in

Moodle 2.7. The new logging system collects more detailed

information about user interaction than the previous system and,

more importantly, provides a standard API for writing and reading

logs, as well as improving system performance.

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

Both registration systems can coexist in Moodle installations with

version numbers 2.7 and higher. However, taking advantage of the

capabilities of the new registration system requires an adaptation of

the different tools, and some of them have not yet been upgraded to

compatible versions. Therefore, the comparison presented in this

study involves the use of different versions of Moodle; it should be

noted that the main objective of this research is to compare tools for

learning analytics, not to address issues related to how logs are stored

in Moodle. The following sections describe and analyze the different

tools, according to the categorization shown in the Introduction of

the paper:

General Purpose Control Panels

Dashboards provide information about student or teacher activity on

the platform and present it in an aggregated and visually enriched

form, mainly in the form of tables and graphs with varying degrees

of interactivity. Dashboards can be applied to different platforms

(Sánchez-de-Castro & Delgado-Kloos, 2012). (Alier & Casany,

2014) or to a specific one (Dimitrova, 2007). These tools focus

mainly on describing the activity performed in an LMS using very

specific metrics, they show some relevant indicators at a glance, but

generally do not provide additional information on how those metrics

relate to each other. The main dashboard application for Moodle is

the Moodle Dashboard. There are other dashboards for Moodle, such

as LearnGLASS or GoogleAnalytics, but these require customization

and mapping of user accounts to external systems and/or direct

coding into the Moodle source code.

The Moodle Control Panel is provided as a block and allows users to

display graphically or literally the result of any query made in

Moodle. When used in standard course formats, the block gives

access to an additional page showing the data generated for the

specified query. There are different options for displaying the

information obtained from queries: tables (line charts, tabular tables

and tree views), graphs (line charts, bar charts, pie charts and

doughnut charts), geospatial charts and maps, and timelines. The

Moodle Dashboard can display the generated data directly, but can

also be combined with other blocks to form a complex and highly

customizable dashboard. It has powerful data filtering capabilities, as

well as functionality to automatically generate data exports

(Dashboard block, 2015). The Moodle Dashboard is compatible up

to Moodle version 2.5.

In addition to the Moodle Dashboard, the default Moodle reporting

tool can also be considered as a control panel. This Moodle reporting

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

tool facilitates the analysis of information about user interactions on

the platform in different contexts, such as site, course or activity. The

reports provide data on user comments, course activity (most active

courses, courses with the most enrolled users, highest participation),

LMS event logs (information about user interaction in the LMS) and

real-time logs (interactions occurring at a specific time), as well as

graphs and statistics on user activity and viewing/publishing actions.

It is also possible to apply additional filters to this information. At

the course and activity level, it is also possible to collect data on

course and activity completion, time spent completing an activity,

and grading information.

Ad Hoc Tools for Learning Analytics

Ad hoc tools are designed for the purpose of monitoring or analyzing

very specific information and addressing a particular need in a

specific context, with a set of defined constraints and conditions. The

main limitation of these solutions is that they generally lack

flexibility and scalability. In this section, two of these tools are

described: (1) Interactions, a Moodle add-on that groups types of

interactions for subsequent analysis, and (2) a web service that

facilitates individual student assessment in teamwork contexts.

Interactions is an add-in that is compatible with Moodle versions 1.9

and 2.0 to 2.3. It is installed as a reporting block that adds

functionality to the default reporting tool, with separate access

permissions. In essence, Interactions adds a library that extends this

functionality, including filtering capabilities, by creating an MS

Excel spreadsheet with two distinct worksheets. The first worksheet

is an exact replica of the log reporting tool's MS Excel file. The

second worksheet processes each record and assigns it to a category

within three different classifications (by agent, by frequency of use,

and by mode of participation). Moodle and eLearning experts were

involved in defining the correspondence between actions and

categories. The final result shows the total number of interactions for

each category for each user on the platform. Since the results are

already in Excel format, it is easy to generate graphs from the output.

In addition, the format allows for seamless integration with statistical

analysis tools such as SPSS. It is relevant to note that the assignment

of each record to a specific category (a record can belong to one and

only one category for each classification group, but can be present in

all groups) is coded in the processing library, which implies that any

modification in those assignments requires modifying the add-in

code.

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

The other tool is an ad hoc web service designed to assess student

performance in teamwork contexts. Based on the work of Fidalgo-

Blanco et al. (Garcia & Sein, 2013) , the web service proposes an

approach to validate data on interactions as indicators of individual

performance in teamwork contexts, based on the comprehensive

training model of Teamwork Competence Training (CTMTC)

(Fidalgo, 2014). The CTMTC establishes how to collect evidence

from three sources: forums, cloud file storage services and wikis. The

system extracts student interactions, which allows individual

students to be evaluated and conflicts to be detected. The tool uses

Moodle's web services layer (Pozo, 2011) and extracts data from

Moodle logs, focusing on posts and forum threads. It works in

Moodle versions from 2.1 to 2.6 (its use in Moodle 2.7 or later

versions would require adaptation to the new registration system).

The tool allows selecting a forum within the course and then

displaying data on student interactions with peers, and has three

different display modes: forum-based, team-based and thread-based.

The tool provides information on the total number of posts in the

forum/team/thread, as well as the number of people registered (the

total number of team members), the average participation of each

student, the list of teams and the complete list of students with their

respective numbers of posts, dates of creation of the first and last

thread, list of threads (with the date of creation) and team members

and degree of participation. In addition, action rules can be

established based on thresholds defined according to the number of

messages (Conde, 2020).

Learning Analytics Tools for Problem Specific Analysis

This category covers learning analytics tools that focus on specific

data and offer a very particular type of representation. These

applications have very specific functionalities and therefore may or

may not meet institutional and personal needs. Examples of tools that

can be used on different platforms in this category are LEMO,

SNAPP, StepUp!, while LMS-specific tools include Moodle

Engagement Analytics, Moodle Learning Analytics Enriched Rubric

or GraphFES.

Our analysis focus will be on two specific tools for social network

analysis: SNAPP (available on multiple platforms) and SNAPP

(available on multiple platforms). (Heathcote, 2021) and GraphFES

(unique to Moodle), both designed to identify students who are not

actively participating and to provide information about social

interactions in the classroom. In addition, Moodle Engagement

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

Analytics will be evaluated. (Engagement Analytics Plugin, 2015)a

Moodle block designed to provide information on students at risk.

GraphFES (Graph Forum Extraction Service) is a web service that

connects to Moodle logs, both old and new standard ones, and

extracts information from all the message boards in a specific course.

Then, all the information collected by GraphFES is processed to

create three different types of graphs: (1) a graph that shows all the

messages aggregated by all users and how they are related to each

other (i.e., a map of all posts and how they are connected and

organized in threads); (2) a graph that connects all users in the course

according to who has read the content posted by others and how many

times they have done so; (3) a graph similar to the previous one, but

in this case, the relationships between users in the course are based

on who responds to whom. Once GraphFES has built the social

network graph, it delivers it as a .gefx file that can be opened in

Gephi. The main idea behind GraphFES is that social network

analysis is most effectively done outside of the learning platform,

using specialized social network analysis tools such as Gephi. Some

advantages and applications of Gephi in analyzing higher education

courses from a social learning analytics perspective can be found in.

(Hernandez, 2014) (Gonzalez, 2020).

SNAPP (Social Networks Adapting Pedagogical Practice)

(Heathcote, 2021) works as a bookmark that extracts information

from message boards in Sakai, Blackboard, Moodle, and

Desire2Learn, and then builds the resulting social network in a Java

application. There are two versions of SNAPP (v.1.5 and v.2.1), and

their functionalities are similar. SNAPP is structured in tabs, the first

three of which are interactive. The first tab displays the social

network graph from interactions and allows the user to manipulate

the graph by filtering, applying different layouts to the social graph

and selecting individual nodes; in SNAPP, the nodes represent

participants on the message board. SNAPP v.2.1 also displays a

timeline of messages posted on the forum. A second tab displays the

values of the number of posts per user in v.1.5 and the main

parameters of the social network (degree, internal and external

degree, eigenvector betweenness and centrality, and network density)

in v.2.1. Finally, the third tab allows exporting the graph in GraphML

and VNA formats in v.1.5, or writing annotations in v.2.1 (export

capabilities are included in the first tab in v.2.1, with the ability to

export to .gefx format).

Engagement Analytics is a Moodle extension that is presented as a

block and has the function of collecting and displaying information

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

through indicators on student progress. As the name suggests, this

block provides users with insight into a student's level of

engagement, which refers to the activities that influence students'

success in an online course. The block provides real-time information

on how students interact with resources and uses a set of indicators

and a risk alert algorithm. This information can be valuable for

teachers to identify at-risk students and make decisions about when

to intervene to prevent student failure. The indicators included in

Engagement Analytics relate to student assessment, forum

participation and login frequency, and it is possible to assign different

weights to each indicator to describe and model the risk of failure in

a personalized way. The indicators are composed of several elements,

and the weights of the elements can also be adjusted. This extension

is compatible with Moodle from version 2.2 to version 2.7 and allows

expansion of the predefined indicators.

Learning analysis frameworks and tools

The fourth category of tools encompasses applications and

frameworks that can be used in various platforms or environments to

investigate different aspects of learning through various visual

representations. Examples of such tools include SAM, VeLa or

GISMO (this review focuses on the latter two).

VeLA (Visual eLearning Analytics) is a framework that uses web

services to extract information from Learning Management System

(LMS) records. VeLA provides various representations of the

information and presents it interactively. For example, users have the

ability to filter, search or dynamically modify the representation of

information. VeLA offers four distinct functionalities: (1) a spiral-

shaped semantic timeline that facilitates tracking user activity on the

platform during specific periods; (2) an interactive semantic tag

cloud that allows users to analyze the most relevant terms and

concepts used in a course; (3) a social graph that displays user

interactions; and (4) a tool to compare and establish relationships

between data stored in the LMS and user activity. VeLA is based on

visual analytics techniques.

GISMO is an interactive, graphical monitoring tool that provides

visualization of student activities in online courses. GISMO is a plug-

in available for Moodle versions 1.9.X and 2.X that allows teachers

to examine various information about students, such as course

attendance, reading materials, or assignment submission. GISMO

provides comprehensive visualizations that give an overview of the

entire class, not just a specific student or a particular resource.

GISMO offers seven different visualizations: access summary,

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

course access, resource access, assignment summary, quiz summary,

resource access summary, resource access timeline per student, and

resource access per student.

MATERIALS AND METHODS

To evaluate the performance of the different tools, this study uses

data from 119 students of an Object Oriented Programming course in

the Software Development Technology Career of the Guayaquil

Institute, Ecuador (Teaching Period 2021- 2022). The course

methodology aims to encourage teamwork among students. The

intensive use of forums, wikis, web resources, videos, quizzes and

assignments in the course makes it a suitable test bed for all the tools

detailed in the introduction section of the paper. The main results of

the application of these tools are detailed below:

• Moodle Dashboard. The latest version of the tool works

correctly for Moodle 2.5. It has been tested on a Moodle 2.6

and no results are obtained from a simple query. With debug

mode enabled, it is also possible to see an error, but no

information is displayed. It is possible that the tool is not

adapted to Moodle versions higher than 2.5.

• Moodle's default reporting tool displays over 122,640 log

records (111,644 are view actions, 9,398 are actions to add

resources, and 821 are update actions). Detailed but raw

information about each action is displayed in a table, and it is

possible to export the results to a spreadsheet.

• Interactions. The plugin does not work correctly in Moodle

versions 2.3 onwards. However, since it only processes data

from the Moodle registration table, it was possible to import

the data directly through MySQL import and process the data

in Moodle 2.1. The result is a spreadsheet, where it is up to

the teacher or course administrator to create graphs from the

data to display information (see Figure 1 as an example) and

detect abnormal levels of different types of activity. The data

can also be analyzed with statistical packages such as SPSS.

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

Figure 1

Charts created in MS Excel using interaction data.

Source: Prepared by the authors

The teamwork assessment tool requires the activation of Moodle web

services. The tool provides a list of links to the course forums. After

selecting them, it is possible to see the participation in each forum, in

a group and individual participation. From this information, it is

possible, for example, to know that the groups working in the

mornings (there is a specific forum for them) have posted 4974

messages with an average of 81.54 per user, and also who is the

person with the most messages (192 messages in this case). When

inspecting a single group (group M9 in this case), the tool reports 990

messages, 6 users, 183 short messages (less than 140 characters) and

807 long messages, 141.43 messages per user, and how participation

is distributed among students (in this case, between 13% and 19%)

(Figure 2). Additional filtering by thread is also possible.

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

Figure 2. Teamwork evaluation tool showing information on student

participation in a forum.

Source: Prepared by the authors

GraphFES provides an interface that requires login credentials,

platform URL and course ID as input data. Once the web service is

active and the necessary permissions have been granted in Moodle,

GraphFES comprehensively collects all information related to

Moodle forum activity from the platform's data log table via the web

service. It then creates two lists containing nodes representing

messages and users, as well as the connections between them that

indicate who has participated in the conversations and who has

replied to whom. The result of this process is presented in the form

of three .gefx files, which can be opened and analyzed in the Gephi

tool. For more detailed information on how to conduct social learning

analytics using Gephi, reference can be found in. (Hernandez, 2014).

Figure 3 provides a visual representation of the three social graphs

generated by GraphFES, covering the totality of users and messages

in the course, without applying filters or using specific node

attributes.

Figure 3. Social graphs of messages posted (9241) and messages

read and responses among users (124, including teachers).

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

Source: Prepared by the authors

Version 2.1 of the SNAPP application does not work correctly in any

version of Moodle and requires setting security exceptions in the Java

runtime environment because it connects to an external source to

perform parsing. To build the social graph, the application loads all

the threads in a forum and processes the HTML content. The problem

seems to lie in the construction of the social graph, and no

participants are identified. Version 1.5 also does not yield any results

for a message board, but allows to analyze individual threads.

Unfortunately, it only works in earlier versions of Moodle (2.1),

where it was not possible to restore course data.

Moodle Engagement Analytics aims to detect at-risk students. The

configuration for this study assigned equal relevance (weight) to

logins, forum participation and assignments. Figure 4a shows the

results of the analysis (at-risk students are shown in red on the left

side). The tool detected 18 at-risk students (probability of failure

greater than 65%). Clicking on the student's name displays a report

explaining why this person is considered at risk (Figure 4a, right

side).

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

Figure 4. Participation block showing at-risk students and the report

of a specific student (left, 4a) and the report of actions in a specific

forum in GISMO (right, 4b).

Source: Prepared by the authors

VeLA uses Moodle log data to show how message board threads

were used, highlighting their primary use in query resolution,

teamwork tasks, and news. In addition, it provides visual

representations of user interactions with peers and resources using

force graphs. This tool offers an integrated experience by allowing

filters and selections to be applied to all views simultaneously.

GISMO offers different visual representations of user interactions.

The example below shows the number of overall actions in the

forums. Students' reading and writing actions are clearly

distinguished, and it is very easy to compare who has the most

reading actions (979) or who has posted the most messages (259).

GISMO can also show students' actions in a specific forum (as shown

in Figure 4b) and in other activities and resources.

RESULTS

From the analysis of the tools, it is possible to observe their strengths

and limitations. However, it is also important to note that the choice

of tool will depend largely on the needs of the users. For example,

Moodle's default reporting tool offers a wealth of information and

filtering capabilities, but the information it provides consists of raw

data, so it offers very detailed information but is not able to provide

meaningful aggregate information about courses. As an example, the

tool cannot answer a simple question such as "How many students

have not yet started a course?", or more complex questions about the

progress of students in a course.

From a theoretical point of view, Moodle Dashboard could provide

answers to these questions, including data visualizations (despite its

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

lack of interactivity). However, this study was unable to test Moodle

Dashboard due to its extremely difficult configuration process and

limited compatibility with the latest versions of Moodle. In addition,

Moodle Dashboard lacks flexibility for custom queries and reports,

making it necessary to use ad hoc tools for particular purposes.

This study has investigated two of these ad hoc tools. Interactions

numerically represents user interactions in a spreadsheet, allowing

customization of graphs and facilitating statistical analysis, and the

teamwork assessment tool has a web interface and focuses on the

analysis and evaluation of student participation in discussion forums.

Both tools solve very specific problems; however, their specificity

makes it difficult to apply them in other contexts or platforms.

The study also described tools designed to address specific problems:

two tools for social learning analysis (three, if the social graph

included in VeLA is considered) and a tool for tracking student

progress and detecting at-risk students. The main difference between

the first two tools is that SNAPP includes a basic social network

analysis module within the platform, although it could not be tested

with the study data due to performance problems, while GraphFES

allows a complete and more detailed analysis using an external

program. In terms of tracking students and detecting at-risk students,

Moodle Engagement Analytics is based on predefined indicators and

facilitates real-time monitoring of a course, allowing teachers to take

action when the system detects at-risk students; a major drawback is

that, despite allowing customization of indicator weights, the

indicators are not intuitive and adding new indicators requires

additional coding.

Learning analytics frameworks aim to overcome the limitations of

the above-mentioned types of tools, and they integrate data, different

functionalities and visualizations, as well as interactive data

manipulation into a single system. Obviously, learning analytics

frameworks are not as well suited to specific tasks due to their general

purpose design. In some ways, these frameworks could be considered

a kind of advanced dashboard that integrates information, but can

also provide very detailed information about courses and students.

CONCLUSIONS

A qualitative analysis of the different tools included in the study

shows that there is a need to add learning analytics capabilities to

LMS such as Moodle within the same platform.

For simplicity and compatibility, some basic dashboard and alert

system would be adequate for this task without the need for

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

additional user training. However, it is demonstrated in this study

how different tools complement each other by adding new

functionalities and that a more insightful analysis of educational data

requires integration, complex visualizations and interactivity, for

which learning analytics frameworks are suitable tools.

A focus on the development, flexibility and stability of the LMS web

services layer would be critical to facilitate the implementation of

these frameworks. In addition, a by-product of a consistent web

services layer is the ability to utilize multiple existing external

specific programs for analysis (as illustrated by GraphFES and

Gephi) that can provide a deeper level of analysis than some basic

LMS add-ons.

Finally, it is felt that the use of complex learning analytics

frameworks is not geared towards students or teachers (whose needs

should be covered by basic dashboards). To reach their full potential,

the frameworks should also integrate institutional and academic

data, and be managed and operated by experts with a learning

platform analyst role. Analysts would then act as "learning

consultants" for the different agents in the learning process (course

coordinators, teachers, students).

REFERENCES

Agudo-Peregrina, A., & Iglesias-Pradas, S. (2014). Can we predict

success from log data in VLES? Classification of interactions

for learning analytics and their relation with performance in

VLE-SUPPORTED F2F and online leraning. Computational

Human Behavior, 542-550.

Alier, M., & Casany, J. (2014). Google analytics for time behavior

measurement in moodle. Information systems and

technologies. 9th Iberian Conference on Information Systems

and Technologies (pp. 383 - 391). Barcelona: AISTI/LaSalle.

Conde, A. (2020). Using learning analytics to improve teamwork

assessment. Computers in Human Behavior.

Dashboard block (February 20, 2015). Dashboard block. Retrieved

from https://docs.moodle.org/27/en/Dashboard_Block

Dimitrova, V. (2007). CourseVis: a graphical student monitoring tool

for supporting instructors in web-based distance courses.

Intercation Human Computer, 125 - 139.

Engagement Analytics Plugin (February 21, 2015). Retrieved from

https://docs.moodle.org/22/Engagement_Analytics_Plugin

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.

Fergusson, R. (12/01/2012). The State Og Learning Analytics in

2012: A Review and Future Challenges. The Open

University. Retrieved from

http://kmi.open.ac.uk/publications/techreport/kmi-12-01

Fidalgo, A. (2014). A comprehensive training model of the teamwork

competence. Int. J. Learn. Intellect. Capital, 1 - 19.

García, J., & Sein, M. (2013). Indicators for the monitoring and

evaluation of teamwork competence through the CTMC

Method. International Congress on Learning, Innovation and

Competitiveness CINAIC 2013. Madrid: Fundación General

de la Universidad Politécnica de Madird.

Goldstein, P. (2005). Academic Analytics: The Uses og Management

Information an Technology in Higher Education.

EDUCASE.

Goldstein, P., & Katz, R. (2005). Academic Analytics: The uses of

Management Information and Technology in Higher

Education. Educause,Colo.

Gonzalez, I. (2020). Applying social learning analytics to message

boards in online distance learning: a case study. Computers

in Human Behavior (in press).

Heathcote, A. (2021). SNAPP: realising the affordances of realtime

SNA within networked learning environments. Seventh

International Conference on Networked Learning (pp. 125 -

134). Lancaster: University of Lancaster.

Hernandez, A. (2014). Usare Gephi per visualizzare la partecipazione

nei corsi online: un approccio di social learning analytics.

Tecnologie Didattiche, 148 -156.

Hernandez-García, A., & Conde, M. (2014). Proceedings of the

Second International Conference on Technological

Ecosystems for Enhancing Multicultutality. Dealing with

complexity: educational data and tools for learning analytics

(pp. 263 - 268). New York: ACM.

Macfadyen, L., & Dawson, S. (2012). Numbers are not enough. Why

e-learning anaytics failed to inform an institutional strategic

plan. Educ. technol. soc.

Pozo, A. (2011). E-learning services in Moodle 2.0. CEPIS Upgrade,

43 - 50.

Romero, C., & Ventura, S. (2010). Educational data mining: a

review of the state of the art. IEEE Trans. Syst. Man Cybem.

Sánchez-de-Castro, D., & Delgado-Kloos, C. (2012). GLASS: a

learning analytics visualization tool. 2nd. International Conference

on Learning Analytics and Knowledge. New York: ACM.

Revista Iberoamericana de la Educación, Vol - 8 No. 1, January - March 2024

Learning analytics as a tool for academic monitoring of virtual students of University Technological

Institutes.