Operations

ADDIE Methodology

Development Phase:

During the Development phase, I focused on automating data capture and distribution in a test environment, integrating multiple software solutions including Arlo, SurveyMonkey, Zapier, Power Automate, and SharePoint. This phase was particularly challenging as it required aligning these commercial off-the-shelf products with IKON's specific business practices. For instance, I used automation software to create custom automations that generated custom emails comprising key event data and registration links from the Arlo database, enabling seamless data transfers and ensuring that learners were automatically granted access to relevant resources.

One of the most complex aspects of this phase was ensuring interoperability between the different platforms. Each system had its own unique set of APIs and data structures, which required careful planning and customisation. I developed several custom scripts and workflows to facilitate communication between these systems, ensuring that data integrity was maintained throughout the process. Additionally, I created training and support resources to help staff adapt to the new system, including step-by-step guides, video tutorials, and an internal knowledge base hosted on SharePoint.

The most difficult stage was the creation, population, approval of, and distribution of the Admin Documents. These are a series of three reports that go to clients with the list of registrants, their feedback on the event, and the trainer's feedback on the event. Before the Digital Development Strategy, the creation of these reports was entirely manual. I first created two automations in Zapier, each with 24 steps. As Zapier was not able to sufficiently handle branching automations, I needed two separate versions of this automation as some events required sending a different survey to the learners, reflecting the more physical nature of some of IKON's training. This proved both ineffective and expensive.

I realised that I would have to move the automations into a single process in Power Automate. This solved the issue with branching: easily handled by Power Automate. The other issue was that Zapier could only create documents in OneDrive, not in SharePoint, meaning that all the admin was being created in my personal OneDrive, which was not a resilient solution. The whole process would fail if I were to leave the company, so I had to transition to the creation of the admin documents in SharePoint instead: this meant that permissions could be granted to enable anyone within the company to access the admin.

Implementation Phase:

The Implementation phase saw the launch of the new system in a live environment. I ensured all users received consistent support during this period, hosting much of the learning content on the IKON SharePoint site and utilising Teams for notifications and updates. I organised training sessions and drop-in clinics to address user concerns and facilitate a smooth transition. These sessions were tailored to different user groups, ensuring that trainers, administrators, and other stakeholders received the specific guidance they needed to perform their roles effectively within the new system.

A significant challenge during implementation was managing resistance to change. Some users were hesitant to adopt the new system, preferring the familiarity of legacy processes. One of the areas of tension was the use of the Arlo-For-Mobile App, which was only available to use on a mobile phone. IKON Trainers had never had to use their own private devices for work purposes. Although we discussed providing company devices, this was ruled out due to costs, and the difficulty of trainers having to carry an additional device with them when they already carry a company laptop. To address this, I employed a combination of formal training and informal support, including one-on-one coaching sessions and real-time troubleshooting for the use of the Arlo app. I monitored usage and provided support where trainers were not using it correctly. I liaised with our external IT company to ensure that the personal devices were covered within the scope of our GDPR and cyber-security protocols, including the Cyber Essentials framework. This approach helped build user confidence and ensured a higher adoption rate of the new technology. I also implemented a feedback loop, allowing users to report issues and suggest improvements, all of which were considered and many of which were actioned. This helped in refining the system and fostering a sense of ownership among stakeholders.

Evaluation and Adjustments:

Following the implementation, I entered the Evaluation and Adjustments phase, where I conducted regular reviews of the system to identify areas for improvement. This included refining workflows, such as updating the certification expiration process to make it more efficient and optimising the learner registration pathway to reduce the time required for enrolment. I used a combination of user feedback, system analytics, and direct observation to identify inconsistencies and areas for enhancement.

One key adjustment was the automation of routine administrative tasks, such as generating attendance certificates and sending follow-up emails to learners. By refining these processes, I was able to free up significant time for the administrative team, allowing them to focus on more value-added activities. I also conducted periodic training refreshers and created additional support materials to address common issues that emerged after the initial rollout. One unexpected outcome was that many of our learners and our clients actually felt bombarded by email reminders. I worked to reduce the number of reminders and reworked the wording and details to reduce the number of emails but increase their effect. Wherever possible, I tried to combine emails coming for different purposes, although this was not always achievable due to emails coming from different software origin points.

The feedback from stakeholders was overwhelmingly positive, however, there were certain areas where we sought to make improvements based on feedback:

• Reduced the frequency of email reminders to reduce learners feeling overwhelmed by communication.
• Created simple 'click here to amend registrations' buttons for commissioners to enable easier ammending of registrants.
• Enabled the sending of Trainer Synopsis hyperlink before the end of the session, so trainers can complete the synopsis of early-finishing events before leaving the venue.

Throughout the entire ADDIE process, my focus remained on ensuring robustness, interoperability, support, compliance, collaboration, and consistency—key values of our Digital Development Strategy. My work in steering this strategy involved a deep understanding of the technical, legal, and operational aspects of learning technology implementation, showcasing my ability to engage thoughtfully with both the broader legislative context and the specific operational demands.

Reflective Comment on Software Integration

The decision to integrate Arlo as our TMS significantly enhanced operational efficiency, though it came with challenges. Initially, I was hesitant to move away from a bespoke PowerApps and SharePoint solution because of the control it provided. However, the tight timeframe and my growing understanding of the complexities involved in building a bespoke system led me to choose Arlo. I was also aware of the vulnerability of creating a system without an outside agency to support me: not only would that make it slower for me to create a product, there was a higher likelihood of errors or bugs, as well as the reduced scrutiny of a dedicated legal support team. By using an external product, these negative outcomes were lessened, but the off-the-shelf nature of the external product brought different challenges.

Early testing revealed significant misalignments between Arlo's intended use case—hosting public events where individuals self-register—and IKON's requirements, where client commissioners register their learners for private sessions. This necessitated me to use automation tools like Zapier, Power Automate, and Azure Logic Apps to create custom workflows that enabled Arlo to meet IKON's unique needs. These included automations that sent registration links directly to commissioners and reminders ahead of events, as well as automations that generated populated reports from data in Arlo and SurveyMonkey. These reports were sent to the administration team for approval before being forwarded to clients, ensuring accuracy and saving hours of administrative effort. The impact of these automations was very high: without them, the Digital Development Strategy would have failed as it would not have met IKON's needs to create a robust, interoperable TMS. However, the creation of these automations, and which software to use, created further considerations and challenges.

Future Developments:

Reflecting on the integration process, I learned the importance of adaptability and creative problem-solving when working with off-the-shelf software. The initial challenges with Arlo required me to think outside the box and leverage various automation tools to bridge the gaps between what the software offered and what the organisation needed. This experience also underscored the value of continuous testing and iteration, as the feedback from early users was instrumental in shaping the final solution.

I have learnt that in-house developed solutions are simpler to control, and enable greater levels of customisation. But it creates vulnerability as only the in-house developer(s) know how to operate and amend the process. There needs to be a balance between external software and internal customisation, as well as clear documentation including visuals. In future, I will try to create wiki-style documentation with flowcharts for any automated processes I develop, so that others within the organisation can learn how to amend or adapt the process. This may be wishful thinking, as a certain level of technical knowledge and ability will always be required, but the process of documentation, and the accountability of doing so achieves the following goals: It reduces vulnerability, enables future developers to take over where I leave off, and it also provides me with a reflective opportunity to look at processes and automations from a higher viewpoint. The ideal pathway to take it to use a mixture of off-the-shelf software; to save development time and reduce single-points-of-failure, and some in-house custom-development which is well documented.

I feel that it is the role of a learning technologist to always stay aware of the latest developments in learning technology solutions and to keep their skills current in the core software. This approach allows us to balance our experience of developing solutions using existing tools and technologies with the advantages of adopting new and innovative software solutions. For example, as AI has risen in prominence in this area, it seems like a logical next step for creating learning materials and solutions. However, I am wary of over-reliance on external solutions, especially when considering issues such as data privacy, intellectual property, and the complexities of integrating AI-driven tools with existing systems. Moreover, there are significant sustainability concerns, such as the high energy consumption and environmental impact of data centres that power AI and other cloud-based technologies.

In the future, a responsible approach as a learning technologist involves not only leveraging these advancements for efficiency and scalability but also carefully evaluating the operational, ethical, environmental and financial implications of their use. It's essential to critically assess whether these technologies truly enhance learning experiences or if they are being adopted for the sake of trend-chasing. By maintaining a balance between innovation and sustainability, we can ensure that we are providing learners with impactful, future-proof solutions while contributing to the longevity and well-being of the educational ecosystem as a whole.

Core Area 1b Technical Knowledge and Ability in the Use of Learning Technology

Technical Integrations Across Multiple Platforms

In my role, I've used a number of different tools and platforms to support operational processes and learning delivery. This section goes beyond the Training Management System (TMS) to look at other technologies I used, and what I learned from using them. These tools included Arlo, Power Automate, Zapier, Azure Logic Apps, and Moodle API integrations, each of which presented different challenges and opportunities for developing my skills.

Additionally, there was an issue with all admin documents being stored on my personal OneDrive, as Zapier lacked the ability to connect to a shared drive, such as SharePoint, meaning that the creation of documents was a vulnerable process (when it worked) as the documents created were meant to be shared files. Over time, I became increasingly convinced that Zapier was not suitable for the long-term resilience of our operation processes, particularly as the costs spiralled with the number of automations.

Solving the Integration Issue

My initial thought was to explore other TMS providers from the list, hoping they might offer a better solution. However, given that Arlo had promised a 'seamless' integration, I became sceptical of other providers' marketing claims. I also revisited the PowerApps solution I had begun at the start of the project, before choosing Arlo. While PowerApps offered a high degree of customisation, I quickly realised that bringing it to the same level of functionality as Arlo would require not only significant time but also considerable upskilling on my part. The learning curve and time investment simply wouldn't have been viable given the project's timelines. This experience highlighted an important balance: the cost of investing time to learn new skills must be weighed against the financial implications, both in terms of salary and project deadlines. Sometimes, even if a solution is technically possible, it may not be the most practical or cost-effective choice.

In the interim, I began to experiment with Power Automate, which was included in our Microsoft 365 licence. There had been several 'flows' created before I joined the organisation that had stopped working. As I worked to fix/replace these, I learnt more about the value of this automation suite. As PowerAutomate is included in the 365 licence, this meant there were no additional costs. I explored how to customise and extend the existing automations within Power Automate, learning how they worked. I studied several online courses on Power Automate via LinkedIn Learning, which significantly improved my confidence. I then began experimenting with duplicating the Zapier automations in Power Automate, with the goal of transitioning them away from Zapier.

I extensively read the Developers' Documentation from arlo, seeing if this would provide guidance on how to 'fix' the issues with the Zapier automations. As I read more on this, I discovered that all the automations relied on API calls the arlo 'back-end'. I had learnt how to make API calls within my Web Development training, so I attempted to employ these skills to develop a solution to the automation issues.

Lessons from Transitioning from Zapier to PowerAutomate

Moving everything to Power Automate took time, but it proved to be a more practical choice. Unlike Zapier, Power Automate could handle more complex workflows, was included in our existing licensing agreements (saving costs), and was better integrated with SharePoint, meaning data could be accessed by others in my absence. This transition taught me a great deal about evaluating the value of different platforms—not just from a technical perspective, but also in terms of business continuity. If I had the opportunity to begin again, I would try not to use Zapier, as although it is 'simple' the restrictions are almost as high as the cost. In an ideal world, a software solution would not require additional automations to enable it to function. If I needed to create automations in the futurewithout access to Office 365, I would consider other platforms.

Exploring Azure Logic Apps

After becoming more comfortable with Power Automate, I decided to explore Azure Logic Apps as an alternative for building automations. Logic Apps had some benefits: it was faster to develop with, thanks to features like the AI-powered code checker, and it provided better error messages, which helped me diagnose issues more easily. I intially built some flows using Logic Apps to link data between Arlo and Moodle, ensuring that only learners who had attended their courses had access to the Moodle resources. However, it turned out that Logic Apps was more expensive to run, and the unpredictable costs made it hard to justify compared to Power Automate, which was included in our licenses. Despite this, the time I spent using Logic Apps gave me a better understanding of automation design and coding workflows from scratch.

Making API calls in other software

One of the more complex integrations I worked on involved linking Arlo with Moodle through API calls in PowerAutomate. My goal was to automate learner enrollments, so that when someone was marked as 'attended' in arlo, their details were automatically updated in Moodle, and they gained access to post-course materials. This integration required me to learn how to work with an additional API in a practical way: understanding the JSON formatting of Moodle, making requests with a different authentication process, and handling the different data structures of both systems. It was a steep learning curve at times, but by the end of the project I had gained a much deeper understanding of how different platforms connect and how to ensure data flows smoothly. I also experimented in making API calls to SurveyMonkey to see if this would work, and whether there were any efficiency savings to be made. However, those flows that made use of the HTTP Request to SurveyMonkey were re-factored as the PowerAutomate native connection was suitable and more stable than using a HTTP request. The HTTP request required the creation of a token, which would only last for a few weeks, whereas the main PowerAutomate to SurveyMonkey connector did not have these expiration problems. I learnt from this that although a custom API request can be made, a native connection should be preferred if it fulfills the purposes of request, as this is more robust and resilient.

Another interesting application involved generating AI images to attach to new client profiles in our CMS. This provided each new lead with a unique visual identity. However, after learning about the high energy consumption involved in creating AI-generated images, I decided to move away from this practice for sustainability reasons.

The most crucial use of the LLM was in triaging new leads that arrived via email. I designed a system where the LLM would automatically score each lead out of ten based on their likelihood to progress to a booking. To do this, I built a prompt that prepended key variables such as the lead's distance from our trainers' locations, the relevance of our current course offerings, and the tone of the request to gauge seriousness. Additionally, the LLM conducted background research on the company submitting the request, helping to prioritise high-potential leads more efficiently. This AI-assisted triage process significantly streamlined how we evaluated and followed up with new opportunities.

Developing a Learner Survey with SurveyMonkey

Another project I led involved creating a comprehensive learner feedback survey using SurveyMonkey. Initially, we considered having separate surveys tailored to different types of courses—online delivery, face-to-face, and train-the-trainer. However, upon reflection, we realised that this approach would result in inconsistent data, making it difficult to compare feedback across different delivery methods. Our goal was to gather uniform, comparable data that could provide insights across all course types.

To address this, we decided to develop a single, unified survey that branched based on the learner’s selected course type. This would allow us to maintain consistency in core questions while still gathering the specific feedback needed for each course format. The branching survey design was more complex to set up and required careful planning to ensure a smooth user experience. However, it ultimately proved more effective for data analysis.

Challenges with System Integration

One of the challenges we faced was integrating the SurveyMonkey results with our automated admin system. While the branching survey approach offered flexibility, it made it harder to link the survey responses to the automated workflows we had in place for course completion and feedback processing. The complexity of branching surveys required more advanced data handling, which was not fully compatible with our existing setup in Arlo and Power Automate. This project taught me valuable lessons about the trade-offs between survey design flexibility and system integration, and the importance of balancing technical limitations with user experience.

Reflection on Learner Survey Feedback

Reflecting on learner feedback was integral to our continuous improvement efforts. The feedback collected through SurveyMonkey highlighted areas for enhancement, such as simplifying the registration process and improving communication about health and safety declarations—explored further on the Wider Context page. This iterative feedback loop has been instrumental in refining our processes and ensuring that our digital solutions meet the needs of all stakeholders.

Reflection on Web-Development for Moodle

Reflecting on the development of content for Moodle, I found that balancing technical precision with user-centered design was crucial. By using tools like HTML5, SCSS, and JavaScript, I was able to build a flexible and scalable platform that enhanced learner engagement and provided a consistent user experience across devices. HTML5 offered a lightweight framework that allowed for seamless updates, while SCSS ensured visual consistency, and JavaScript added dynamic, interactive elements that guided learners through their journey. Accessibility was also a key focus, ensuring that the platform met WCAG guidelines and could be easily navigated by learners with diverse needs, including those using screen readers.

While I experimented with tools like H5P for interactive content, I found it difficult to align these elements with corporate branding and styling. Although tools like Articulate or Adobe Captivate could have provided more advanced content creation options, they came with a steep learning curve and a significant cost. Given the time constraints and the need to maximise resources, I chose to rely on my existing skills in web development, which allowed me to build content with zero outlay and full control over the design and functionality. This approach not only saved time and money but also allowed for a more customised, consistent learning experience.

Reflection on Batch File Development

On reflection, this project highlighted the balance between developing quick, functional solutions and the time required to refine and perfect them. The script achieved its primary goal, but turning it into a more polished, user-friendly tool that could be used by the design team without manual intervention would have required further development. Additionally, this experience demonstrated the importance of investing time in understanding tools like ImageMagick, which, when combined with scripting, can provide significant efficiencies in automating repetitive tasks.

Overall, this exercise emphasised the need for careful planning and testing to ensure that even small automation scripts are robust and scalable. Had I had more time, I would have liked to turn this into a standalone application that the team could use, which would not only resize images but also address any folder-structure issues automatically. It was a valuable learning experience in understanding the trade-offs between creating a functional tool and building something that is fully refined and optimised for long-term use.

Next Steps in Developing Technical Skills

In the future, I want to continue building my skills with APIs and automation. As we expand our use of Moodle, I plan to use Power BI and/or other data dashboard tools to better visualise the data we're gathering on learner progress and engagement. With the ability to track key metrics, such as completion rates and quiz scores, in real time, we can make data-driven decisions to enhance learning outcomes and address any gaps. Power BI’s integration with other Microsoft tools also offers the potential for seamless reporting and insights at both the learner and organisational level.

Beyond data visualisation, I aim to further explore advanced automation solutions. Automating routine processes, such as steamlining the learner enrolment process, certificate to be stored in Moodle, and better communication workflows. These could significantly reduce administrative load. This would allow us to focus more on improving content and learner experience rather than managing repetitive or duplicitous tasks. By building on my current expertise in Power Automate, I intend to investigate more complex automations using Azure Logic Apps and explore AI-powered automations that can intelligently adapt to different scenarios. These advanced tools can help us scale our operations more efficiently, ensuring long-term cost-effectiveness.

Additionally, I am interested in expanding my understanding of API integrations beyond Moodle, exploring how other learning management systems, such as Canvas or Blackboard, could be integrated into our ecosystem. By becoming proficient in cross-platform API interactions, I can ensure smoother data exchange between different systems, enhancing both learner and administrative workflows. Another area I am keen to explore is the use of AI and machine learning within automations, particularly in predictive analytics, to anticipate learner needs or identify trends that can inform future course development and personalisation strategies.

To achieve these goals, I plan to continue professional development through online courses, industry certifications, and collaboration with technical experts. Staying up to date with the latest advancements in automation, data visualisation, and AI will enable me to future-proof our systems and keep us at the forefront of learning technology. As the landscape evolves, I aim to integrate more sophisticated tools that not only streamline our processes but also enhance the overall learning experience for our users.

Engaging Stakeholders

One of my primary responsibilities was to ensure that all stakeholders were on board with the new systems being deployed. This began with early engagement sessions, where I gathered input from trainers, administrative staff, and even learners. During these sessions, I conducted surveys, interviews, and workshops to understand needs and concerns. For example, administrative staff were primarily concerned with the usability of the new system, while trainers wanted to ensure that they could access learner data easily without having to spend time checking details were correct. By addressing these concerns early on, I was able to tailor the deployment strategy to meet their expectations.

I also organised regular meetings with the training team to discuss progress, challenges, and upcoming milestones. This approach ensured that everyone was kept informed, and it fostered a sense of ownership among the stakeholders. The feedback loop established during these meetings was crucial in identifying any issues early and making the necessary adjustments.

Automation allowed for timely delivery of emails and reminders, reducing the risk of missed communications and ensuring that all participants were well-prepared before the course began. It also freed up valuable time for administrative staff to focus on more critical tasks.

To ensure that all those interested could access the content of the course communication, I created the Course Communication Overview Map. This was split into Pre-Course, Post-Course, and Other, so that everyone could understand the flow of information. The origination source of the communication was listed, with a clear outline of the conditions. Crucially, each communication event had a content document that contained the content of the communication, so that this could easily be updated by anyone across the team.

Of course, the updating of this map and the associated content documents necessitated actual updates to the automation origination source: Arlo, Moodle, or the automation software—Zapier or Power Automate.

Change Management and Addressing Resistance

Resistance to change was a significant challenge during the deployment of new learning technologies. Many users were accustomed to the legacy systems and were hesitant to adopt new processes. To address this, I used a combination of formal training and informal support. I provided clear demonstrations of how the new systems would benefit them in their roles, such as reducing manual administrative tasks or improving access to learner data.

In addition, I worked closely with the trainers to ensure that the new technology was integrated into their daily routines in a way that was manageable. For example, the introduction of the Arlo-For-Mobile App required trainers to use their personal devices, which was initially met with resistance. I addressed this by ensuring that our IT policies covered the use of personal devices and that proper GDPR and security measures were in place. I also provided one-on-one support to trainers who were struggling with the app, which helped to build confidence and increase adoption rates.

Providing Training and Support

To ensure a smooth transition to the new learning technologies, I provided tailored training sessions for different user groups. These sessions included hands-on workshops, step-by-step guides, and video tutorials that were accessible on demand. I recognised that different users had varying levels of technical proficiency, so I made sure to adjust the content accordingly. Trainers, for instance, were provided with more in-depth training on how to use features related to learner data tracking and assessment, while administrative staff received training on managing enrolments and accessing reports.

I also hosted drop-in clinics, both virtually and in-person, to support users who needed additional help. This was particularly useful for those who were less confident with technology, as it provided them with the opportunity to ask questions and receive real-time support. These clinics also helped me identify common issues that users were facing, which informed the development of further training materials.

Supporting the deployment of this technology involved more than just technical work. I played a crucial role in strategising the rollout of our eLearning platform, working closely with the training team to ensure a smooth transition. This process included conducting training sessions for staff, preparing support materials, and establishing a feedback loop with learners to continually improve the platform. I also drafted content for our clients that was released in a series of posts on our website: here and here.

Reflections

Reflecting on the deployment process, I learned that effective support requires a balance between technical knowledge and interpersonal skills. Engaging stakeholders early and providing ongoing support were key to a successful rollout. One of the biggest lessons was the importance of adaptability—being willing to adjust the deployment plan based on user feedback made the implementation smoother and increased overall satisfaction.

Another lesson learned was the value of creating multiple support channels. Not everyone learns best in a workshop setting, and providing a range of support options—such as video tutorials, written guides, and live support—helped ensure that all users could find the help they needed in a format that worked for them.

Reflecting on this journey, I recognise that the successful implementation of learning technology is not just about choosing the right tools but understanding how these tools fit within the broader educational context. Each tool needs to serve a purpose, whether linked to enrolment and registration, gathering of feedback/survey data, or the hosting of learning resources. Balancing the technical aspects with pedagogical considerations and stakeholder engagement has been a rewarding challenge. There is significant progress still to be made, particularly around increasing the capacity of our systems and better processing the vast quantity of data we are now producing.

Next Steps in Supporting Learning Technologies

Moving forward, I aim to enhance the support provided by incorporating more advanced analytics to track system usage and identify areas where users may need additional help. This data-driven approach will allow me to be more proactive in supporting users and ensure that the learning technologies are being utilised to their full potential. I also plan to develop more self-service resources, such as an expanded FAQ section and interactive tutorials, to further empower users to solve problems independently.

This process has not only enhanced my technical skills but also deepened my understanding of how technology can be harnessed to create impactful learning experiences. In the near future, I hope to apply data analysis processes to our databases to better develop our operational processes, improving operational efficacy and providing learners with better metrics on their performance. I expect this development to occur naturally as we build more learning courses within Moodle and have more learners progressing through their learning journeys. You can read more about this on the Next Page - Teaching and Learning.