Evaluation: Final Report

Evaluation: Research or Evaluation

Evaluation: Types of Data

Learning from the PBL Process

When I reflect back on how a project should be run, I realise that I wasn't going into the detail needed nor was I focusing on the correct things i.e. the driving question, the motivating entry event, the scaffolding of the assessments, a detailed plan, and the culminating event. Basically everything.

In another module I designed a digital learning experience that I thought would work as a good project, but I see now from my PBL  work that I was not creating a thorough enough scaffold for my students to use to achieve the goals I had in mind.

The Driving Question has been one of the biggest eye openers for me with this module. I wasn't finding a question that was motivating enough for the students to want to go the extra mile with. On top of this, a weakness in my project has been the entry event. Even up until the peer review, my entry event wasn't very strong, however I think I have found an excellent video that discusses the rather outrageous abuse biotechnology companies are doing to farmers that will hopefully insight students to take action.

I have found the project writing experience very similar to doing a Bernie Dodge webquest. Obviously there are differences but there are elements of a webquest design that I think are rather redundant and repetitive, in the same way, there are parts of the project site that ask for repeats of content they have already been done. When I do this process again in the future, and I will do it, I will cut out the extraneous bits and link the relevant resources to the appropriate writing as I have done in this project. In this way, any of my colleagues viewing the content will have everything they need to run the project in as efficient a format as I can manage.

I have built other project sites in the last year and it is clear that I need to return to them and completely review the motivational attributes and assessment requirements in them. A proper culminating activity needs to be designed so that the students take the tasks more seriously as an authentic audience.

Post project reflection

My colleagues will play a substantial role in the review of the project. They will highlight where elements of the project went well and what needs to improved. This will likely occur as they happen so I will keep a writing record of any discussions we have. As I want to see my colleagues engage in the use of a department blog this year, perhaps this can be a mechanism by which they record their observations.

The students will be asked to provide a response via a survey using Likert scales. This will allow me to quantify their opinions of the project concept and how it went. My students aren't shy in coming forward so no doubt I will hear about their concerns as they arise.

The experts and parent audiences that come into contact with our students will be asked for their opinions of the work and interactions they see, this will allow us to tune these elements for the future.

All of my work goes through multiple iterations and evaluative feedback. The students are instrumental in what and how my colleagues and I present our course. Trusted students will be sought out on a weekly basis and asked for their opinion; they know that their voice has power and that we will react to enhance their learning experience. One of my biggest concerns is that the students will find the project too much for them to handle along with their other subject content. If this is the case, we most be monitoring the students' concerns very regularly.

Digital Learning Experience

I have been considering developing this Plant Growth Project for some time. From the perspective of an Instructional Designer, there was an identified need that required solving; my students need to engage with the Biology on a personal level and through the integration of their assessed course material, the use of real scientists like Prof. Dr. Normah, tech enhanced assessment, and redefinition of the classroom by using technology to visit places virtually that we cannot visit in real life. While I still need to incorporate some elements into the project, this iteration is certainly at a point where it can be released to my students and used. I have previously developed the Google Sheets used to record the students scores to present the students' assessed performances as Progress Charts for each of the skills, these need incorporating but I felt were beyond the scope of this particular project. Since the embedded GSite is the actual project my students will be using, any enhancement I make will appear here automatically. Visit the actual site here.

Technology Rollout

Click here for the Full GSheet. It is view only; you will need to make a copy if you wish to input any data in the evaluation column

I designed this tech roll-out rubric to encompass all the elements of a major tech roll-out such as a new VLE integration or a large integration such as that of Google Apps for Education. I have no doubt that finer divisions of the rubric scores could be achieved; I would expect that an evaluator would assess a firm categorisation and add appropriate comments to elucidate why they have chosen this category and any actual differences that are apparent in the integration. Comments that identify what has been done well and what needs to be improved would be expected.

While some rubrics expect an evaluator to indicate which assessment level they are selecting by ticking a box, these rubrics tend to have a separate sheet with descriptors for levels that require the evaluator to flick back and forward to. I have designed this rubric to incorporate both on one sheet using Google Sheet's new conditional formatting to highlight evaluation descriptions based on a drop-down selection of the evaluation for that element.

The Graph tab gives a visual representation of the evaluation; this is built as the evaluation column is completed.

Strategic Change Models: Educational Technology Goal

For the Educational Technology Goal project I have created an integration plan for a real e-Portfolio project. The task required that I identify obstacles to success; these are inferred in the design.

The Google Sites ePortfolio for Target Setting and Enrichment activities will be designed, trialled and implemented for Year 10 students by the end of Term 3, 2014.
The flowchart below outlines the individuals and groups with a stake in the success of the implementation of this new system. Adherence to the timeframes outlined will be essential to the success of the implementation of the ePortfolio for the Key Stage (Year 10 and 11) for the start of the next academic year.

Final Instructional Design Project

Grant 1.4 - Final

The final grant proposal has developed from a series of submissions and reviews of each section of the proposal document. The feedback from the instructor has led to continuous revisions leaving this final document requiring little revision from the previous draft; I am pleased with the result.

This document is an example of whole school technology integration and represents a final artifact for the School Technology Coordination certificate. It represents professional knowledge and the skills of writing grant proposals for a particular audience. The evaluation plan highlights elements of assessing and evaluating learning goals with the collaborative support of subject matter experts.

ID Formative Evaluation

The evaluation plan outlined is based on Kirkpatrick’s Four-Level Evaluation model on page 11 of Streamlined ID (Larson & Lockee, 2014).

School Evaluation Summary

Evaluation | Survey

This was an interesting task; I only wish I had had more time to do a more thorough job. Perhaps if this task had been assigned earlier on in the semester, I would have been able to do the job to the level I will need to justify improvements to my bosses! That being said, I now have some data that verifies what I have thought about "SCIS" in the past.

My school has been in the process of trying to achieve the NAACE Mark which is a British technology maturity certification. It is rather similar to the maturity benchmarks we used in this task. It has not been my responsibility to evaluate "SCIS" against the NAACE Mark criteria but I have been asked to give my opinion in the past. I was not as positive in my assessment one year ago as those that were performing the evaluation and I was clearly being seen to be looking at the evaluation with a "glass half empty" perspective. After completing this task, I am confident that my previous assessment was accurate and that "SCIS" has a lot of work to do.

It was interesting to see the differing perspectives offered by the members of staff I asked to collaborate with me on this task. The Systems Adminstrator, the Assistant Principal for Development, and the Head of IT curriculum all completed the survey I prepared. After bringing all of their responses together I was able to get a much wider picture of the state the school is at. After given the evaluation to these staff members they too were interested in the differences in perception.

The elements that I have identified that need attention: training, new technology use, and assessment, will likely need 2 to 3 years of planned development to get them up to Intelligent level. As I mention in my evaluation, a culture shift will need to occur where it comes to the attitudes of the staff. I regularly visit an American international school and play sport with the staff there. Through this interaction I have had the opportunity to talk with the Americans regarding their attitudes to training. It is clear that the requirement to go through re-certification is a big plus for institutes that have this in place. The staff know they have to meet the standards or they will not be able to teach; their jobs depend on them being up-to-date.

In the past I submitted the suggestion/plan to the leaders of "SCIS" that the school take a more serious view of the attendance at training and the development of the skills needed to work in a 1 to 1 environment. I made comparisons with other educational systems such as the US and Japan but it was decided to continue with training being "optional". The idea of making staff do certification seems completely unrealistic! Whilst "SCIS" employs staff on a 2 year contract basis, they are not prepared to use this pressure to insight change.

Before I return to school I intend to fully compare the benchmark with the NAACE Mark. Had I had the time to be more thorough for this task, I would have asked department and faculty heads to complete the survey along with a sample of students, parents and governors where the survey is appropriate. It was clear to me after I got the responses back from the three other staff members with responsibility for technology integration that perspectives are very different. We will need to thoroughly go through each element and evidence each part from the students perspectives right up to the governors if we intend to succeed with the NAACE Mark.

AECT Standards 2012: Professional Knowledge and Skills

AECT Standard 4 - Professional Knowledge and Skills: 
Candidates design, develop, implement, and evaluate technology-rich learning environments within a supportive community of practice.

Indicators:
• Collaborative Practice - Candidates collaborate with their peers and subject matter experts to analyze learners, develop and design instruction, and evaluate its impact on learners.
• Leadership - Candidates lead their peers in designing and implementing technology-supported learning.
• Reflection on Practice - Candidates analyze and interpret data and artifacts and reflect on the effectiveness of the design, development and implementation of technology-supported instruction and learning to enhance their professional growth.
• Assessing/Evaluating - Candidates design and implement assessment and evaluation plans that align with learning goals and instructional activities.
• Ethics - Candidates demonstrate ethical behavior within the applicable cultural context during all aspects of their work and with respect for the diversity of learners in each setting.


NOTE: Parenthetical page references are to Educational Technology: A Definition with Commentary (2008, A. Januszewski & M. Molenda, Eds., Lawrence Erlbaum Assoc.)