1a) An understanding of the constraints and benefits of different technologies.

I firmly believe in the judicious application of technology; I run screaming from "technology for technology's sake". I am amazed by how often lecturers will ask me for "some tech" to "liven things up" or "increase engagement", and I've learnt the best way to respond to such requests is to start by booking some time to sit down with them to work out the issues they're hoping technology will solve. Often the answer is as pedagogical as it is technological - whilst, say, introducing polling software to a long lecture could well keep students on their toes, and take some measure of interest and understanding in the room, it might be that a flipped classroom approach, in which key concepts were made easily available to students in bite-sized videos, short podcasts, or clearly-labelled reading, allowing for half the usual amount of lecture time, with the remainder of the session spent doing group work, or creating responsive content. I find that the constraints and benefits on a tech-by-tech basis are usually clear enough, but that the weightiest part of the work is establishing whether or not they are appropriate to the issue at hand.

I’m often asked to weigh up one platform against another, either at procurement process, or comparing institutional platforms and purposes. Evidence 1 (below, end of page) is an example of this, comparing polling systems available at my previous institution.

To outline a more complex example of considering the benefits and constraints of technology in my current role, incorporating leadership in learning technology, and exemplifying a more strategic approach, I would like to outline my approach to the choices around whether or not to use the new-to-institution platform, Wiseflow, for departmental assessment, in part, in whole, or as a pilot, to communicate my work in this area.

Description:

In 2020, as I arrived at my current institution, I was involved in supplier meetings and requirement-gathering for the acquisition of a platform to manage online exams during Emergency Remote Teaching. Having participated in platform procurement processes in my previous role (as outlined in Section 4: Communication and Working with Others), my recent experience in the importance of understanding stakeholder requirements served me well, and I immediately began looking at the ways in which assessment was structured and managed in each of my departments.

I concurred with the selection of Wiseflow for exams, and, by the time the platform had been acquired, in winter 2020, had access to the User Testing panel, and the opportunity to thoroughly explore the constraints and benefits of the platform.

The institution decided to approach deployment of the platform via a Project Team – which, again, matched my previous experience. I felt well-placed to support both the project, and deployment of the platform, but was cautious about where it might suit my Faculty. Identifying specific user-end benefits and constraints of the platform was my first move.

Benefits of Wiseflow (as procured in 2020):

• Facilitated blind and double blind marking as standard.
• Incorporated Turnitin Similarity Reports.
• Clear role distribution and separate tabbed views for academics, administrators, and students.
• Support for a variety of assessment types, including multimedia assessments, quizzes, and oral exams.
• Strong administrative focus, including the promise of detailed connectivity between assessment platform and Student Records System, which would, in time, significantly reduce admin workload.

Constraints of Wiseflow (as procured in 2020):

• Complex marking routes requiring more “clicks” at academic side than previous platforms, such as Turnitin or Moodle Assessment.
• Did not allow marking in Turnitin – no option for pre-saved Quickmarks.
• Role distribution severely limits capabilities, meaning that academics who are usually able to run assessments cannot, as the Manager role negates the mandatory student anonymity setting.
• Not all assessment types to be enabled in first year of adoption.
• Significant workload from administrative team in creating and managing assessments.
• Significant power to the Manager role, difficult to fully train and support during rapid deployment.
• Whilst the road map for platform adoption included many workflow improvements, these would not be in place during 2020/21.

Use of the platform was mandatory for central exams, but it transpired my Faculty had comparatively few of these, therefore the main route to central workflow support and development would not be particularly well-used. There was a strong central push to acquire and demonstrate engagement in each Faculty. The intention was for Wiseflow to form a long-term solution for digital assessment for the institution as a whole, and Faculty Learning Technology Leads were tasked with identifying areas in which this would be most useful.

In order for my Faculty to participate in any significant sense I would need to find areas not only which would benefit from adoption, but also which would represent disciplines and processes not necessarily found elsewhere in the institution that the platform would need to support when it came to full institution adoption in the future.

Peculiarities of Arts & Humanities:

• Either a well-established route to online assessment, usually using Turnitin, OR a recent ERT-forced conversion to online assessment from paper, using Turnitin in a “light touch” way (1-2 members of a departmental admin team managing inboxes and submissions, and distributing marking to academics).
• A majority of “simple” setups – 3,000 word essays that dd not require either blind, or double blind marking.
• Comparatively low numbers of teaching administrators.
• Several departments where academics where solely responsible for the setup and management of their assessments.

As I was looking for an area that Wiseflow might be of interest to, I was contacted by the Art School, who had just been informed that they would not be able to use the setup they had worked with in March 2020, as the internal team who had constructed it had conflicting work requirements in the year ahead. They requested support for the large-scale submission of digital portfolios of multimedia work – and I immediately wondered if Wiseflow could provide the solution.

I tested the multimedia upload “flow”, and felt that it would be straightforward enough. I proposed to Digital Education that this would be a useful early adoption project to explore, and that the constraints and benefits of the platform, as well as the limitations of the first iteration of Wiseflow in our institution, just so happened to perfectly align with the requirements of the assessment. Further, there was no other institutional platform which could provide the large, integrated submissions of multimedia at the scale required. I began by outlining the stakeholder requirements (Evidence 2: Stakeholder Requirements, below), and shared these with the Project Team to see if it would be possible to progress.

It took significant communication (Evidence 3: extracts from such communications at Proposal stage) to illustrate my own understanding of both the scale of the project, the constraints and benefits of the platform, the precise nature of the work I personally would need to undertake to conduct this whole-school adoption of the platform, and a significant amount of planning, testing, demonstration and reporting was required to consistently reassure the Project Team that this was a viable endeavour.

I further detail the academic and student outcomes of this in Section 2b. (An Understanding of your Target Learners) but in short, 100% of students successfully submitted their digital portfolios into Wiseflow across 2020/21, and even at the return of physical assessment in 2021/22, the process was retained for Masters students, as it was deemed such a success.

I learnt a great deal about the suitability, and the lack thereof, of Wiseflow for a variety of assessments during the support of the Art School with their assessments, and this allowed me to better understand where else in my Faculty it would and wouldn’t be suitable. My main conclusion was that “Where departmental administrators already contribute significantly to the creation and management of assessment, where blind and double blind marking are incoming requirements, and/or where digital assessment has been minimally, or unsuccessfully used in 2020/21, Wiseflow may well offer significant improvements and benefits.

In 2021/22, based on this conclusion, I recommended the Philosophy Department progressed with the adoption of Wiseflow for all assessment, and, with the assistance of our new Digital Assessment Advisor, this came to pass over the year (Evidence 4: screenshot from a Case Study of Wiseflow for Philosophy, confirming my involvement).

Reflection:

In completing this section I found this to have been a most useful area of my work to have discussed, as it not only illustrates my understanding of the constraints and benefits of Wiseflow for assessment, but also a time at which my ability to demonstrate that understanding to stakeholders, including the department with the requirement, and the institutional project team, was key to the not only to the success of the assessments, but to the platform adoption happening at all.

My previous experience with working with Project Teams, the need to consistently evidence and demonstrate understanding, and, more than anything, the importance of illustrating the ways in which participation will support and contribute to the required project outcomes, was certainly a factor in both the way I approached information-gathering, and the designing of my workload and stakeholder interactions. Further than the platform requirements here, I needed to assess, process, and communicate the benefits and constraints of platform choice both to the users, and to the Project Team. I believe that the deciding factors were that a) the platform was the correct one for the assessment b) I was able to demonstrate a lack of any viable alternative c) my ability to offer a rich pilot of the platform for a user type and use case not currently represented across the rest of the institution.

As can be seen from the proposal, I had strong ideas about the workload and workflows that would be required to successfully bring the platform in to the Art School, and understood that the Art School’s unique setup in data and records terms would conflict with the overall roadmap for the project. I encountered significant resistance from the Project Team, and had to restate these plans multiple times, and actively demonstrate them in numerous meetings in order to convince and communicate the fact that these differences from other programmes would, firstly, not have derailing consequences either for the Art School or for the Project, and that secondly, these differences would highlight crucial factors about both the technological and pedagogical constraints and benefits of the platform for both administrators and for academics. More than anything, however, my conclusion was that a truly robust understanding of not only the known benefits and constraints, but the possibility for unknown ones, was key to successful implementation.

These takeaways have since allowed me to instigate and support the successful deployment of the platform for the Philosophy department, and have also helped me decide where not to recommend, and indeed, where to recommend against the use of the platform in other areas.

Demonstrating that I am not only present to actively promote and deploy central systems, has, I think, had the additional bonus of supporting relational trust between myself and academic colleagues, who, in some spaces, had initially feared having “yet more technology pushed upon” them. Where no benefit could be seen, I was up front and confirmed this, and have been able to focus instead on either supporting existing workflows and platforms, or, where business-as-usual is stable on this front, using my time to support other areas.

Evidence:

1. Polling Software Comparison:

2. Stakeholder Requirements PDF:

Click to view:

3. Extracts from my proposal for use of Wiseflow for the Art School:

4. Screenshot of Draft Wiseflow Philosophy Case Study:

1b) Technical Knowledge and Ability in the Use of Learning Technology

I have worked, and continue to work with a broad variety of learning technologies across my roles, including:

Software Names(s):	Type of Tech:	Roles Supported:	My Involvement:
Moodle.	VLE	Academics, Students, Professional Services, Core Support Team.	Structure, programme/module design, configuring both front and back ends. Designing assessment, workflows, marking support, use and usability, accessibility, reporting. Troubleshooting. Strategic use and development.
Turnitin, Wiseflow.	Assessment and Feedback.	Academics, Students, Professional Services, Project Team.	Assessment design and workflow, marking methods, platform tour, similarity report guidance, academic integrity. Troubleshooting. Strategy and deployment.
Mentimeter, Vevox.	Student Engagement and Feedback.	Academics, Professional Services.	Platform demonstrations, reporting, workflow, integration with Moodle. Question design, academic engagement, feedback design.
Echo360, Planet eStream, Panopto, MediaCentral (internal video hosting platform).	Lecture and Learning Capture.	Academics, Professional Services.	Lecture capture, pedagogical design of learning capture, technical design of learning capture. Integration with Moodle. Strategic direction and workflow.
CampusPress, Mahara.	Blogging and Portfolio.	Academics, Students, Professional Services.	Training and demonstrations. Use cases, case studies, reporting. Configuration and customisation. Modelling good practice. Communicating with CampusPress to support use of widgets and plugins, as well as troubleshooting.

I also utilise a variety of resource and repository sites, such as subscription sites like Box of Broadcasts, and Talis Aspire reading lists. Whilst these are supported by the Library in my current institution, understanding how they work, the types of content available, and the ways in which they can be integrated with Moodle, I can better support the holistic approach to the use of available technologies, looking at the student journey as a whole, rather than as a series of “stepping stones”.

Description:

Previously in my career, much of my technical ability revolved around troubleshooting. I worked both first- and second-line support for LogicaCMG, which gave me an excellent background in technical support and understanding, as well as the basics in software structure and problem-solving. As I moved into learning technology, I found myself most enjoying solving technical Moodle issues, finding the one of many settings that needed to be altered, or the line of code which has been accidentally pasted somewhere it shouldn't have been (Evidence 1: Exchange with academics, below).

My granular front and back-end understanding of Moodle, its settings and capabilities has also served me well amongst my peers. Whilst several of the other Faculty Learning Technology Leads at my institution are similarly well-versed and practiced with it, others have come from either different VLEs, or less hands-on or technical backgrounds. I feel that I have been able to bring my technical knowledge and support to the FLTL community, and that these skills both immediately conferred a sense of my ability, and the type of colleague I might be, and strengthened our overall capacity to exist as a rounded, informed community of distributed leadership. Evidence of these technical skills, and the support I have been able to develop with colleagues, is provided below (Evidence 2: comment on my technical skill with Moodle from fellow FLTL Fiona Harvey).

Another key element to my approach is that I've learnt to be up front where I need to go away and research new technologies further, since coming to understand how often academics and service staff will be confronted by a new "software solution" to one thing or another when away at conferences or events. Often the latest thing they're interested in duplicates, or imitates, something we already have available at the university, so meeting such requests for a particular piece of tech and responding with further engagement and recommendations falling within our existing toolkit is a very positive part of my work.

The request for support from an academic running a Publishing course (Evidence 3) neatly encapsulates both these points: a typical request for support both with the technicalities of assessment design, as well as for a technical skills presentation to the students. Participating in these sessions further gives me an opportunity to see Faculty students actively engaging with the technology, which confers significant information around student digital capabilities, responses, and interests, that supports my day-to-day work.

Reflection:

Whilst the pedagogical and strategic elements of my current role are its defining characteristics, I am grateful that I am still “close to the tech” enough to support and resolve technical issues as part of my day-to-day work. These smaller, individual issues provide an excellent opportunity for me to strengthen my relationships with academic staff across my Faculty, to swiftly resolve something that I am very familiar with, which has been a huge source of frustration and inconvenience to the academic. The years I’ve spent managing support inboxes have given me an efficient, cheerful approach to resolving technical issues that, I believe, has been vital to developing the Faculty perception of my skills and depth of understanding with learning technology, as well as enhancing their appreciation of my approachability and willingness to support them at the level that is required.

Often, those who have had a small technical issue resolved will approach me to see if I am able to support with a more pre-emptive query, e.g. for assessment design, which I don’t believe they would necessarily have done without prior experience. My “smaller” technical skills are perceived as evidence of my deeper understanding of the technology, and, where I can couple the experience of exercising these with a positive working environment – especially if I then go on to take an interest in the academic’s pedagogical purpose, their course design in Moodle, or their use of a particular activity for teaching and learning.

My willingness to listen to colleagues from a variety of areas, disciplines and backgrounds about their technical issues, and my dedication either to fixing them myself, connecting them with central resources for support and resolution, and further providing them with pre-emptive support and resources to prevent any recurrence of the issue, if possible, has served me well thus far. On reflection, I feel that my technical skills with learning technology, much as my instinct can often be to play them down to emphasise my strategic, and pedagogical capabilities, carry tremendous value, securing my ability to work across a range of learning technologies in a way that truly inspires the trust which permits strong leadership and influence at a Faculty level.

Evidence:

1. Exchange with academics, resolving a technical issue where code was pasted into Moodle:

2. Comment from Fiona Harvey, Faculty Learning Technology Lead for Social & Historical Sciences:

3. Support request from an Academic on a Publishing Course:

1c) Supporting the deployment of learning technologies.

Using an institutional plugin to enrol students from the Student Records System into Moodle.

Description:

My current institution has a quirk of Moodle enrollment, which requires all courses to be “mapped” to our student records system. This mapping needs to be manually activated in order for the system to sync with Moodle overnight, and for the students to be enrolled on the course accordingly.

This process was covered in some detail during Emergency Remote Teaching, as students only had their Moodle pages to go on. In many cases, departments supported earlier mapping than usual to give students a chance to communicate with their academics, and fellow students, and the process was reasonably straightforwards, with only a few support queries and occasional technical niggles to sort out. However, I realised at the start of 22/23 that I was receiving rather more baffled communications from academics than usual, and that somewhere the deployment of this process had come unstuck.

Firstly, I found that new academics during Emergency Remote Teaching would take a Connected Learning Baseline course, which introduced them to the mapping concept. This course is no longer supported, and we have returned to pre-ERT scenarios. In my Faculty, this meant a wide variety of things, including, in some departments, the expectation that the academic would map their students…but no communication from the department that this needed to be done.

Attempting to get ahead of this, as part of a session I scheduled for Module Leads and Teaching Admin (those with the power to enrol students in Moodle), I demonstrated how to map students to a Moodle course, and included links to institutional resources supporting it. However, I continued to receive queries from newer staff, who had perhaps yet to be added to the Faculty mailing list which was used to invite Module Leads to the training session, or indeed who had so much to cover in their first days in their new role that they had yet to get to the finer points of the VLE.

With this in mind, I created a (very) brief resource, essentially the quick summary of actions to take that I would put in an email to support an academic asking how to action enrollment, and shared it with the departments I knew expected academics to action this.

Reflection:

This has been one of the more complex deployments of a piece of technology, first and foremost because the consequences are so dramatic: not only can students not see the course if they aren’t enrolled, but it will take a further overnight sync for them to be added, creating a very tangible inconvenience to both students and academics, and potentially damaging the student perception of either their new academic, their course, or Moodle itself. Consequences which include an inability to use technology often result in the belief that there is something wrong, or inherently fallible around the platform, so managing expectations around this issue is in itself an important, and delicate matter. Secondly, as this process is not a new necessity, but an annual one, it already exists in a tangled and inconsistent way, meaning that any universal approach risks alienating those who have already established familiarity with it, and at worst, appearing to “nag”. Working to ensure the up-front work with this plugin has illustrated to me the complexity around communicating absolutely crucial information at the right level in all areas of the Faculty.

As illustrated by the evidence below, some academics who have experience with Moodle don’t imagine that institutional setups could be so different, and therefore “don’t know what they don’t know”. I have had to change my approach to include mention of the need to activate mappings in as many spaces as possible, as pre-emptively as possible (i.e. from the moment at which this activity is released each year). (Evidence: Teams post, reminding Teaching and Learning Admin). I no longer hold any assumption that anyone, regardless of Moodle expertise, or length of time in the institution, is aware of this process. I have also slightly shifted my tone to suggest that this is a particularly common issue, in part to prevent those who are well-aware of the process from “switching off”, and also to encourage colleagues to mention it to, and indeed to support, each other where the matter arises.

Further, in many other Faculties this mapping process is carried out entirely by administrators, so there is very little central awareness that there could be an issue with students simply not having been mapped to a course. First line IT support for students who can’t see modules historically didn’t always check the course was mapped, and on reflection, I realised communicating the importance of checking this element at module level to the IT support team was important.

The actual clicks required to activate mappings are so few, and it’s a relatively easy process, but it isn’t intuitive. The fact that you need to Turn Editing On in the module to be able to see the enrollment block, for example, has caught many people out, and that’s now one of the first things I mention. I also realise that academics, especially new academics, aren’t yet familiar with the types of codes that define their cohorts, so don’t necessarily know how to check things. This wasn’t something I was familiar with either, so, after speaking with my line manager, I requested elevated student records access, which will, firstly, give me greater visibility and understanding of these formats myself, and secondly allow me to support academics “on the spot”.

The complexity of our Moodle setup has shown me many additional complications, too. There are several modules on which many academics teach one particular section, but are unsure who the responsible lead is for enrolling students across the whole. There are non-taught modules which students either manually enrol in, or have access keys to, as these modules, such as glossaries, or handbooks, don’t have corresponding student record enrollments.

Overall, this comparatively small functionality has such broad implications and reverberations around some of the less standardised approaches to using Moodle for teaching and learning – it continues to highlight areas that perhaps aren’t necessarily streamlined, or understood sufficiently. It’s shown me that, whilst simple directions, well deployed (for example, sent to all module leads by a Head of Department), can go a long way to reducing the number of issues arising, there is no single answer at this point, simply because, at the higher level, there are so many setups, configurations, and historic practices which space has not been made to address in terms of workflow. Certainly, this is not a deployment I have yet resolved with total success, but it is one that continues to teach me so much about the nature of the act of deployment itself.

Evidence:

Two emails from academics in response to support with mapping their students:

Blog post summarising the enrollment process:

https://reflect.ucl.ac.uk/abbiwrites/2022/09/29/activate-mapping-how-to-enrol-your-students-into-moodle-from-portico/

Screenshot of a Teams reminder to Teaching and Learning Administrators: