Illustrations by: Marco Politano, Appraiser Team, Agnė Rapalaitė

Illustrations by: Marco Politano, Appraiser Team, Agnė Rapalaitė

Beyond the full circle

Self‑assessment of trainers in the youth field

  by Snežana Bačlija Knoch and Marinela Šumanjski 

26/06/2019

 

How do trainers look at themselves, gain feedback from others and support one another? Snežana and Marinela take us on a journey of discovery – all praise to them!

Evaluation, feedback and assessment are well-known and widely used concepts in the youth (work) training world; and youth trainers themselves mostly understand the need for comprehensive (self‑)evaluation and in general strive towards constant improvement. From the survey Practices of reviewing trainer performance done by the Guild of Trainers in 2017, 97% of 114 trainers agree to the extent that regular review of trainers’ competence leads to better performance, and two thirds of them review their performance after each activity.

Having many different ways to perform the assessment, the usual practice in the youth training field is often based on external feedback, which is collected through rating and end-of-an-event evaluation. Structured self-assessment is not so common and neither is comprehensive interpretation of the assessment insights. Therefore, it could be argued that standard practices sometimes lack depth and holistic approach.

 The full circle (or what is 360 degree feedback)

That lack is where the 360 degree feedback model comes in. Interestingly enough, 360 degree feedback has been in use since the 1940s, and has evolved to become a management tool used in both the private and public sector. 360 degree assessment is an approach and a tool that gives a person the opportunity to perform self-assessment, as well as to receive comprehensive feedback from a number of sources relevant for their work.

In the youth training context, the complete approach is not practised that much, but elements of it have surely influenced how trainers’ competences are assessed. When used in full, this multi-source assessment is usually based on four groups of feedback providers:

  1. trainer’s supervisor/manager/employer/contractor
  2. trainer’s colleagues/co-trainers
  3. participants of the educational activities that the trainer worked on
  4. and the trainer themselves

Hence the name, as the person gets the feedback from the circle 360 degrees around them. This method allows the trainer to understand how their competences, based on observed performance, are viewed by others.

 

 Why?

It is very unlikely that trainers have built their professional paths by behaving randomly. This leads to having some level of self-awareness regarding their strengths and weaknesses. At the same time, honest and reliable feedback is necessary to put one’s perceptions to the test, recognise hidden strengths and point out overlooked issues.

Systematic feedback can provide insight into knowledge, skills and behaviours, bringing to the trainer a necessary reality check of where they stand at the moment. It helps professionals to understand themselves and how they interact with others, but also to understand the impact they have on those around them.

In addition, it can support the development of practices to achieve better recognition of people working in the training field, and can contribute to the overall quality and impact in youth work training. The work of trainers becomes more accountable and there is a solid reference point for professionalisation.

 And why not?

There have been many arguments on why 360 degree feedback is not the most suitable tool for assessment of one’s abilities, especially if the method is not performed correctly. These general questions can cause an obstacle when using 360 degree feedback:

  • How to adapt questions to different groups of respondents (language used and what exactly we ask feedback for).
  • How to choose the most relevant feedback providers.
  • How to minimise the misunderstanding of questions and responses.
  • How to analyse and read the results data.
  • How and who should use the feedback results (only the trainer or their contractor also).

Furthermore, it takes a lot of time and effort, but it still cannot stand by itself as a successful assessment system. It is meant to be used as a positive addition and only an aspect of the overall effort. Having such a comprehensive process can be overwhelming to take on and digest sometimes, especially when it comes to analysing the results and planning follow-up. Understandably, this approach is not for everyone, whether you don’t think it is worth your time and effort, not effective enough, misleading or simply not your cup of tea.

 Beyond the full circle: 360 +1 of the Youth Department of the Council of Europe

In 2016, the Youth Department of the Council of Europe decided to create a tool based on the 360 degree feedback approach and named it “361 degree feedback”. Jonathan Bowyer of View From Here, together with Council of Europe trainers, developed it for participants of the long-term Training of Trainers for Youth in the Council of Europe (TRAYCE), as part of their learning process.

Connecting all the dots, 361 was based on the competency framework of TRAYCE and collected feedback from four groups of relevant actors in the trainer’s work through survey. Answers of around 50 questions organised into eight main competence sections gave a snapshot of the views and perspectives of a range of people who have seen trainers at work. The report was then visually presented through graphs, written answers, a summary “radar” chart at the end of each section and a few word clouds.

 Wait, how do you mean +1?

The extra degree (1 in 361) was added by the developing team to put emphasis on the debriefing and reflecting about the feedback with the mentor, which supported the whole learning process of trainers. That was one of the most important aspects of the whole experience, which also responded to one of the main weaknesses of the general 360 degree approach – lack of support in analysis and no plan set following receiving the feedback. With support from the mentor, the trainer was able not only to deconstruct the results and cross-read the answers coming from different feedbackers, but also set up the plan on how to improve and strengthen competences further.

What do people who used 361 have to say about their experience?

 DANI PRISACARIU: The 361 tool was very useful for me. I used it in an important period of my life, one with intense professional growth and development as a trainer and facilitator, so receiving such valuable information from my peers, participants and collaborators was a great resource to shape the direction I was going in. I wouldn't have been able to have this information otherwise. Although, it was not an easy job to put myself out there, I felt vulnerable and exposed. Another thing that I found challenging was the wording of the tool. For people that are not used to the youth sector I believe it was not very easy to understand all the nuances. Surely, I would like to have the possibility of using this tool on an ongoing base, for example every year, and ask for feedback from all those I worked with during that year.

 MARINELA ŠUMANJSKI: My experience with using 361 goes from the anxiety of going through such a complex process and fearing the result – through confusion who to ask for feedback, questions on why people are not responding enough, is it too boring, too long or too complex – to excitement to find out how people I work with/for see me, surprise to see how critical I am to myself, deciding that I actually want to be a trainer and finally making the clear plan on what exactly I should work on to become a better one.

 CIHAN KILIC: It provided a structured feedback from different perspectives. It is usually hard to ask for feedback, but 361 provided an opportunity to put it in a structured way and not going through the burden of personal level interaction. It was challenging to make the boss at the organisation use the platform to give feedback and reminding people again and again to fill in the feedback form. I did not have much time to go through and analyse it. It was challenging to find people who knew how to give feedback or so to say whose feedback would be valuable in a sense that it would be constructive.

Yuliya Stankevich gave us a short testimony as the person who experienced both tools for trainers’ assessment – 361 and Appraiser. Here is what she said:

 YULIYA: 361 degree assessment is something I would recommend to any trainer, both experienced and beginners. In 2016 I have gone through 361 assessment system as participant of TRAYCE and this year through 360 system while testing “AppRaiser” platform as member of the International Youth Work Trainers Guild.
Though it caused some anxiety for me both times, in the end there were a lot of things that helped to see my strong sides as trainer. For example, insights caused by talks with mentor (in TRAYCE) and in peer trios (within IYWT Guild).
With 361 system my mentor’s support was most valuable. And with “AppRaiser” it was a fascinating and engaging process of self-assessment, reflecting and watching my digital wheel of competencies gaining colours step by step.

Now you must be wondering what is this Appraiser that Yuliya talks about. Stay with us and find out in the next part of the article.

 Beyond the full circle: 360 +2 Appraiser (Competence assessment platform for trainers)

The story of Appraiser started in 2016, during the bi-annual Bridges for Trainers meeting, among a group of trainers associated with the International Youth Work Trainers Guild (Guild of Trainers). While standing in front of the competences area’s posters of the ETS competence model for trainers, they imagined that, in order to motivate trainers to commit to professional development based on the competence model, a tool would come in handy. The decision was made to develop a digital tool that would be accessible, intuitive and inviting! What followed was a Key Action 2 project of the Erasmus+ Youth in Action programme of the European Union, which was supported by JUGEND für Europa and now, almost three years after, the tool is starting to gain shape.

Besides being enchanted by the colours and charts of the ETS competence model for trainers, the choice for Appraiser being developed around competence-based assessment is that the team felt that it brings the needed frame and language (albeit complex at times) that everyone involved in assessment can share.

360 degree assessment/feedback/review was chosen based on experience in the youth training field, which indicates that trainers oftentimes select those people that they would like to get the feedback from, which does not necessarily provide a realistic picture of their competences. Hence, there was a need to ensure coherence and a holistic approach with Appraiser.

Appreciative inquiry is a change management approach which is based on affirmation and recognises the best aspects in people, affirms past and present strengths, assets and potentials. Through the survey and interviews it was confirmed that the majority of trainers look at the things that they are not satisfied with and the Appraiser team wanted to make a shift in this process.

And finally, design thinking is the main approach behind development of the tool. It is an innovation process, which puts people (users) and their needs at the centre of the creative process and makes sure that the team gets to know their potential users (in this case trainers), their habits and preferences, before starting the developments. In Appraiser it is a non-linear process, managed through a well-planned design cycle: 1) observation and research; 2) ideation; 3) prototyping; and 4) building and implementation. And it was precisely the design-thinking approach that led the team to reach out to both the theory behind the different elements and to the continuous needs assessment (through surveys, interviews and testing) of Appraiser.

 That’s great, but where is +2 in all that?

… you might be wondering and rightfully so! Well, one of the main pillars of the work of the Guild is peer support that comes in the form of trios, which are formed yearly and whose purpose is to support collegial professional development. So, when thinking about the missing part of 360, which is how to process all those amazing insights, charts and quotes that come from self-assessment and feedback, the team decided to go back to the trios and make them another essential pillar of Appraiser. The trios are formed by three colleagues that have not worked together previously (or not very often, at least) and they are there to support each other in: going through the 360 process; processing results and outcomes of the 360 process by asking appreciative, deepening and clarifying questions and pointing out biases and critical judgments; planning further professional development steps; and in being there for each other, empathising and truly listening.

 

 So how does Appraiser work?

We can tell you about it in detail, but you’d better see it for yourself. Just click on Appraiser, create your profile and dive into the world of self-assessment. Now, we say self-assessment, but in reality trainers (again through surveys, interviews and testing) indicated that they very often start their assessment path by collecting feedback. Whichever entry point you choose, it is important to collect the 360 degree journey and then some more. We feel that those +2 make all the world of difference and that they are a true step towards professional development.

Here are the core Appraiser elements (in no linear order):

  • creating your profile as a trainer;
  • performing self-assessment based on one or more competence areas (including rating, providing comments, uploading evidence);
  • asking for feedback based on one or more competence areas, connected to training events and to a selected group of participants, peers and contractors;
  • receiving visual charts of self-assessment and feedback;
  • having trio support to process 360 outcomes and plan further professional development;
  • having tools and suggestions for further professional development;
  • having the possibility to share (parts of) 360 outcomes;
  • having guidelines (both technical and pedagogical) to support each of the elements.

Agne Rapalaite tried out this tool and this is her enthusiastic response at the end of the process:

 

 Where are we now? (some of the insights from the process so far)

If you ever meet someone from the Appraiser team, they would be very happy to share all the excitements about the process so far and amazing plans lined up for the future. At the same time, there have been challenges along the way.

Time is an issue. It is! Trainers say it loud and clear in the surveys and interviews. Contractors and participants say the same. Allocating time for doing a comprehensive self-assessment and providing feedback is not so easy and will continue to be a challenge.

We (the trainers) love freedom and our non-formal ways. We do! Therefore, some of us are still debating whether a competence-based approach is the way to go.

We (the trainers) love cards, Post-its, markers, drawings. We love face-to-face conversations. We do! During Appraiser LAB where we tested all the phases of Appraiser, trios sat together, bonded and surrounded themselves with the competence cards, their notebooks, different special pens. This is all to say that understanding the digital tool as an essential element of Appraiser might still take time to be adopted.

We (the trainers) are not all the same! No, we are not! (and hence all the above statements are probably irrelevant). Trying to find an approach that would fit each trainer and those who will be providing feedback is quite a challenge. However, we had to find some basics and this is what we are trying to do with different levels of assessment.

We are excited and so are most of the people who took time to test Appraiser at different stages. And in case you would like to know more, you can reach out to us at: appraisal@iywt.org.


At the end, every jump into assessment depends on the contexts you work in and defining that context is very important, as is defining the major reason for wanting to do this. Try to make of it a valuable learning experience and keep a positive approach to using it in the trainers’ community. Importantly, both initiatives, 361 and Appraiser, were developed as part of trainers’ different types of learning journey, and they are not planned to be used in more hierarchical contexts where trainer assessment is part of a contractual review process. Be open to the feedback and extract the constructive aspects of it for further development, but also see it as an appraiser of your achievements. Lastly, keep in mind, dear reader, that we are still continually developing and questioning things in the training field and would be very grateful to have feedback and different views from colleagues on this topic.

  Do you have comments on the article?  
Use the Feedback form to give your comments and/or visit our Facebook page!

 

 

 

 

 

Issue 28

European Youth(work)   
   Training

 

 

Snežana is an educator, consultant, International Youth Work Trainers’ Guild board member and a wannabe creator.

 

Marinela is a trainer and facilitator in the youth work field and the biggest fan of non formal education. She was proud to be a participant of TRAYCE. Co-editor of Coyote #28.