Tag Archive: Evaluations


If you think the last thing you’d want to do is deal with the aftermath of your camping trip,

THEN you are wrong.

The last thing you want to do

is deal with the aftermath of someone else’s camping trip:

Believe me!

Having said that, as I gazed over the tent city that occupied my backyard, all I could think about was project management practices and training.  I thought, “what did I get myself into?”  I then reminded myself that hopefully I would be remembered as a good team-player and mother… But, let me explain from the beginning.

My boys went on another one of their camping trips with the 8th Newmarket Scout Troop. [Incidentally they are in the Venturer program not the younger  Scouts program.] This trip involved a 4+ hour drive  to the Bruce Peninsula National Park (further than most 2 day camps) and there was a bleak weather forecast (typical for this Venturer Group).  Mother Nature didn’t disappoint: the scenery was incredible, the rain torrential and they set ‘a first’ for their group: they went to the laundry to dry some sleeping bags and clothes.  They had a great time and returned home safely: tired, wet and dirty. My husband dashed off to catch a flight and then all the boys disappeared: wet camping equipment still in our garage.

Exams loomed for the boys and the timing to get assistance from the Group was very tight. The weather forecast was grim: rain all week. Replacing moldy tents would be several hundred dollars: What we were going to do?

Then a ray of hope: actually of sunshine. The weather radar looked like we may have a few hours of sun before the rain started again. Luckily my work schedule was flexible enough to accommodate opening up numerous tents, a shelter, ground sheets and tarps. However, I wasn’t prepared for the mud covered everything. Needless to say this was a multi-step process: open up the equipment, hose off the mud, dry the equipment, re-inspect and start all over again if necessary.  The messy work was completed when the boys arrived after school: they worked as a team and folded up all the tents, tarps and shelters.

Now what does this have to do with Training?

Many focus on preparation and delivery of teaching, training while they ignore what should happen in the aftermath of training (or any project).

1-Schedule the time (and personnel) to deal with the ‘aftermath’ of training. This goes beyond the ‘tear-down’ and storage of equipment and extends into the training material. One needs to leave all material ready to use in the future. So, remove unwanted content (like a wet sock in a tent) and fill the holes/bridge the gaps. Just like with camping: You won’t remember where the hole was when you are in a hurry to set up the next time around.

 2-Ask for and examine participant feedback:  it may surprize you. One may think that given the inclement weather the Venturers would have had a marginal time: on the contrary they had the BEST time battling the elements!  [I will examine ‘the art’ of feedback in detail in upcoming posts.]

3-Take the time to reflect as the deliverer. What did we do that we would do again? What should we do differently? Jotting down these ideas right after training will capture them; otherwise, you may not remember specific challenges (or victories) until you encounter them again.

4-Exploit all windows of opportunity, even if the timing is inconvenient or the task at hand is ‘not your job’. Yes, I was tired and I didn’t want to do all the work for my boys by drying out the tents for them. On the other hand, I probably saved the Group a lot of money as they would have had to replace all the tents.

So much can be gained by taking the time after a training session, or any project. Applying the lessons learned and fixing the challenges faced will have a profound positive impact on results going forward. Just ask my boys who had to deal with someone else’s dirty dishes the night they were packing to go camping. Now to get them not to leave dirty dishes in my kitchen sink…

Until next time, this is ‘Scoutess Anne’ , as the boys call me…

One of my boys is going into High School in September. I’ve been frustrated trying teach him how to study.

Harvard opting out of final exams is not going to help my case.

Although my teenager made the Honour Roll, he comes home with tests and exams… and excuses. He shows me the test and tells me how ambiguous (and stupid) the questions are. Unfortunately, many times I agree with him. He argues that the facts and concepts that his teacher is trying to test “don’t matter”. Then he argues that the question(s) really shouldn’t count… so his mark is really out of 90 marks and not out of 100. It is hard to provide a counter argument on the days when he is making sense.

So maybe there is some merit to placing less emphasis on finals…

What about you? Did you ever write an exam where some professor placed some obtuse unimportant topic on the exam? Did that exam really showcase the breadth and depth of your knowledge to your professor?

Or did you, like me, dread the type of question, not the topic? I dreaded multiple choice questions: they were not my forte. Secretly I called them ‘multiple guess’ questions. I swear there is a black art to answering multiple choice exams. (I always thought when in doubt you should answer ‘c’; however, I found evidence to suggest that ‘b’ is the most common answer.) So, maybe the best way to answer multiple choice tests is to follow the advice of a friend:

the wisest way to answer a multiple choice question is  to chose the answer that you think your examiner views as being correct (versus choosing the right answer).”

If we step back for a minute…

WHY TEST in the FIRST PLACE?

Years ago apprentices were trained and tested via projects during the course of their apprenticeship. Today,  carpenters, electricians and other trades continue that practice; while, doctors intern and lawyers article. (And if your results are not acceptable in that period of applying your ‘craft’, you do not graduate.)

When you think about it, what a daunting task of trying to ‘test’ the knowledge of all your students. That is where exams come in. Exams were designed to provide a sample the student’s knowledge on a wide range of topics and subjects. And exams are less prone to cheating than ‘projects’ are.

Chester E. Finn Jr. & Mickey Muldoon have posted a blog post discussing the dilemma with Harvard, an educational trend setter, opt-ing out of Final Exams.

Some will say that other student work products — term papers, especially, but increasingly multimedia projects, too — are better gauges of learning than cumulative exams. Associate Dean Stephanie H. Kenen recently stated: “The literature on learning shows that hands-on activities can help some students learn and integrate the material better.”

Do exams really test your knowledge and ability? Do exams really determine your skill level? Are projects that demonstrate applied knowledge better gauges of learning?

HIRING WILL EXAMINE YOUR KNOWLEDGE PORTFOLIO

Actors audition and artists provide a portfolio so people, including potential employers, can ‘see or experience’ samples (and ranges) of their talent, character and viewpoint.

Knowing that some Universities are opting-out of final exams, Hiring Directors and Recruiters may change the way they recruit, when they are recruiting new University/College Graduates. I expect that over time they will place a higher priority on work experience (including co-op terms) and profession related extracurricular activities.

On the other hand, the hiring of experienced employees will change to place an emphasis on what I call their ‘knowledge portfolio’. No longer will the post secondary undergraduate education, certification and experience be the only items that employers will look for. Just like an artist, a professional will need to highlight their talent in a knowledge portfolio: a collection of sample accomplishments and pieces of work that represent their character and viewpoint. A perfect platform for the sharing of one’s knowledge portfolio is provided by social media. Content can be created or curated (and commented on) or groups can  collaborate on content. This way written works (from blog posts to white papers) to multimedia works (informal videos to  formal podcast demonstrations) can be shared.

Others have talked about the idea of  ‘knowledge portfolio’:

  • The guys at drawer.com talk about a knowledge portfolio as something that you invest in over time. They have good pragmatic advice about building skills and making “learning a habit”.
  • An older Agility Essay posted by Rick Dove talks about the importance of having the right knowledge at the right time & right place… He discusses other aspects to consider and the article is a good read.

These are all very good perspectives and  good advice.  However, by using the term ‘knowledge portfolio’, I was trying to emphasize  the word portfolio. I believe people need to SHOWCASE evidence that their opinion, viewpoint and work are important.

Maybe the word ‘knowledge’ should be replaced with ‘professional’. I found an article by Alison Doyle that talks about the assembly and value of professional portfolios. Alison’s view is closer to what I wanted to say. Having said that, I do prefer the term ‘knowledge portfolio’.

CERTIFICATION WILL CONTINUE

I don’t see professional certifications disappearing. AND I don’t see professional certification exams disappearing either.

I think back to high school where a smart colleague of mine got a higher French mark than me. How was that possible? She spoke French like a 10 year old just learning: pausing mid-word. It was SO painful to hear her speak. It was possible for her to get a high French mark because she could write a French exam and the written exams had higher weighting than any oral ability. So, on paper she looked brilliant, although she couldn’t order a meal in Paris!

While that is a simplistic example, I can ‘see’ training and certification evolving to emphasize more practical and applied skills.

Some certification processes are already like this: you have to perform on the job (and write up several different projects where you’ve applied your expertise) and you have to be interviewed. Here the interviewing process becomes an exploration of the breadth and depth of your experience and talent… but it is dynamic. It is not a one size fits all static ‘test’ like an exam is.

BUSINESS ‘Learning’ will EVOLVE

For those who are delivering instruction as a business (or for a business), you can see the trend already. There is more emphasis on the application of knowledge via ‘practicals’: exercises, discussions, assignments and performance support deliverables.  ‘Tests’ at the end of a session will be replaced with assignments, demonstrations and observation… all performance indicators. The bottom-line is that training will cost more to implement, but the emphasis will be in the right place: on performance not just the ‘recall of facts’.

To those in the Instructional ‘Biz’, (Kirkpatrick’s) Level 2 (Knowledge) and Level 3 (Behaviour) Evaluations may morph. (Testing ‘knowledge’ in a Level 2 evaluation is typically done by self-assessment or a test. Whereas, evaluating the change in ‘behaviour’ in a Level 3 evaluation is accomplished by observation.) Perhaps the Level 2 & 3 evaluations will become one OR Level 2 will change to see if the knowledge is applied.

ADVICE

For those of you in the instructional business, I suggest that you:

  • emphasize the application of the concepts you are teaching. Be practical, how are people really going to use and apply what you are teaching them?
  • keep an open mind regarding evaluating how much your students have learned. As instruction evolves, so must the way in which we evaluate the impact of instruction.

So, should we tell our kids about Harvard?

I’m NOT! (And I don’t think it would occur to a fourteen year old to read his mother’s blog… so I think I am ‘safe’.)

Even though I think the emphasis (to ‘test’ knowledge) will become more practical and applied, I will still work with my son to prepare for and write exams. I will also encourage him to:

  • apply his knowledge. Even informal teaching solidifies knowledge, even if it is teaching your buddy, Paul, Calculus.
  • collect and document profession related experiences and testimonials.
  • build a ‘knowledge portfolio’ leveraging social media to showcase his expertise.

I’ll keep you posted on my son’s career aspirations. With his persuasive and sometimes argumentative nature, maybe he’ll go to Law School…

Then again, he’ll have to write his LSAT

I better keep trying to teach him how to prepare for and write an exam, despite how ‘stupid’ he may think the questions are.

Until next time…

*********************************************************************************************************

Flickr Creative Commons Image by Olga.