1
00:00:00,400 --> 00:00:02,680
This is an ADLAB PRO video.

2
00:00:02,760 --> 00:00:06,080
ADLAB PRO is a project funded
by the European Union.

3
00:00:06,160 --> 00:00:10,000
This video is made available to you
on a Creative Commons licence.

4
00:00:10,840 --> 00:00:13,960
Hello. I’m Louise Fryer,
from Utopian Voices.

5
00:00:14,040 --> 00:00:15,920
In this ADLAB PRO video

6
00:00:16,000 --> 00:00:19,200
I’ll talk about evaluation
in the AD of live events.

7
00:00:19,280 --> 00:00:22,280
This is Unit 7 in Module 3.

8
00:00:22,560 --> 00:00:26,160
The aim of any evaluation is
to check you’re meeting your goals.

9
00:00:26,240 --> 00:00:29,960
You need to be sure
what AD is trying to achieve.

10
00:00:30,120 --> 00:00:33,240
Opinions differ.
For some people the aim is

11
00:00:33,360 --> 00:00:36,360
to make everything that
is available to the sighted audience

12
00:00:36,440 --> 00:00:38,720
available
to the people with sight loss.

13
00:00:38,800 --> 00:00:42,000
For others it is
to enable users to follow the story,

14
00:00:42,080 --> 00:00:44,560
for others to give users
an equivalent experience

15
00:00:44,640 --> 00:00:46,640
to that of the sighted audience.

16
00:00:46,960 --> 00:00:50,120
There are two types of evaluation.
What we are discussing here

17
00:00:50,200 --> 00:00:53,960
is “formative” with the intention
of improving the final result.

18
00:00:54,040 --> 00:00:56,520
This is different
from a “summative” evaluation

19
00:00:56,600 --> 00:00:59,240
that assesses that final result.

20
00:00:59,960 --> 00:01:03,160
For training purposes,
evaluation helps the AD student

21
00:01:03,240 --> 00:01:07,200
know how well they’re doing.
It applies to the professional too,

22
00:01:07,280 --> 00:01:09,240
as we are always learning.

23
00:01:09,400 --> 00:01:12,280
Some AD companies such as VocalEyes
"VE" have embedded evaluation

24
00:01:12,360 --> 00:01:15,160
throughout the AD process.
VE also carries out

25
00:01:15,280 --> 00:01:17,720
an annual assessment
of each describer.

26
00:01:17,880 --> 00:01:21,040
The scholar Schjoldager
used the following criteria

27
00:01:21,120 --> 00:01:24,280
to evaluate the performance
of interpreting students:

28
00:01:24,360 --> 00:01:28,000
Comprehensibility,
delivery, language,

29
00:01:28,080 --> 00:01:30,840
coherence and plausibility,
and loyalty.

30
00:01:31,040 --> 00:01:34,160
It seems logical that AD users
need to be able to understand

31
00:01:34,240 --> 00:01:36,440
what is going on
in the performance

32
00:01:36,520 --> 00:01:38,760
but research shows
that comprehensibility

33
00:01:38,840 --> 00:01:42,360
and enjoyment
or engagement are not related.

34
00:01:42,440 --> 00:01:45,000
The importance depends
on the purpose of the piece.

35
00:01:45,080 --> 00:01:49,400
For example, understanding a joke
is not the same as laughing at it.

36
00:01:49,800 --> 00:01:52,840
Comprehensibility
has been coupled with delivery.

37
00:01:52,920 --> 00:01:55,440
Not only does lack
of vocal clarity or mumbling

38
00:01:55,560 --> 00:02:00,200
affect comprehensibility, but also
aspects such as segmentation,

39
00:02:00,280 --> 00:02:03,440
gaps between words
and phrases, and prosody,

40
00:02:03,520 --> 00:02:08,560
including intonation, pitch and pace,
all help to shape meaning.

41
00:02:08,800 --> 00:02:11,960
There are many “deviations”
from good delivery including:

42
00:02:12,040 --> 00:02:16,240
inarticulate speech,
pauses, hesitation, false starts,

43
00:02:16,320 --> 00:02:20,160
irritating noises, repetition,
excessive self-correction,

44
00:02:20,240 --> 00:02:24,200
unconvincing voice quality
and monotonous intonation.

45
00:02:24,440 --> 00:02:28,800
In addition to word choice, it is
important to assess word order.

46
00:02:28,880 --> 00:02:31,840
“She extinguishes the candles,
followed by her husband”

47
00:02:31,920 --> 00:02:33,640
is less ambiguously as

48
00:02:33,720 --> 00:02:36,520
“Followed by her husband,
she extinguishes the candles”.

49
00:02:36,600 --> 00:02:40,040
That way only the candles
are extinguished.

50
00:02:40,520 --> 00:02:44,000
Accuracy is not as easy
to define as you might think.

51
00:02:44,120 --> 00:02:48,920
Inaccuracy in AD includes
misnaming something or someone,

52
00:02:49,000 --> 00:02:53,320
using an inappropriate verb
so walk instead of run.

53
00:02:53,400 --> 00:02:56,360
Misreporting,
saying something or someone is there

54
00:02:56,440 --> 00:02:58,720
when it’s or they’re not.

55
00:02:58,800 --> 00:03:02,760
Omission or censorship, not saying
something’s there when it is.

56
00:03:02,920 --> 00:03:05,360
Users care about accuracy,

57
00:03:05,440 --> 00:03:08,920
density, wanting neither
too little nor too much,

58
00:03:09,000 --> 00:03:11,600
delivery should be
engaged and engaging

59
00:03:11,680 --> 00:03:15,360
and technical aspects
such as sound quality.

60
00:03:15,560 --> 00:03:17,960
An evaluation sheet is available
on the ADLAB PRO website

61
00:03:18,040 --> 00:03:20,240
for you to use. It can evaluate

62
00:03:20,360 --> 00:03:23,200
not only your peers’ performance
but also your own.

63
00:03:23,440 --> 00:03:26,240
When receiving feedback:
Assume good intent.

64
00:03:26,360 --> 00:03:28,680
Remember,
they aren’t criticising you.

65
00:03:28,760 --> 00:03:32,920
Their aim is to improve the AD
for the benefit of AD users.

66
00:03:33,000 --> 00:03:36,920
When giving feedback be
specific, honest and kind.

67
00:03:37,000 --> 00:03:39,400
We are all human.
We all make mistakes.

68
00:03:39,880 --> 00:03:42,680
That was a video
about the evaluation of live events,

69
00:03:42,760 --> 00:03:45,320
with me Louise Fryer.

70
00:03:46,200 --> 00:03:48,360
The preparation
of this presentation

71
00:03:48,440 --> 00:03:52,040
was supported by ADLAB PRO.
Audio Description:

72
00:03:52,120 --> 00:03:55,760
A Laboratory for the Development
of a New Professional Profile

73
00:03:55,880 --> 00:04:00,160
financed by the European Union
under the Erasmus+ Programme,

74
00:04:00,240 --> 00:04:03,520
Key Action 2,
Strategic Partnerships.

75
00:04:03,600 --> 00:04:04,800
Project number:

76
00:04:04,880 --> 00:04:13,240
2016-1-IT02-KA203-024311.

77
00:04:13,800 --> 00:04:17,120
The information and views
set out in this presentation

78
00:04:17,200 --> 00:04:20,240
are those of the authors
and do not necessarily reflect

79
00:04:20,320 --> 00:04:23,400
the official opinion
of the European Union.

80
00:04:23,480 --> 00:04:26,840
Neither the European Union
institutions and bodies

81
00:04:26,920 --> 00:04:29,440
nor any person acting
on their behalf

82
00:04:29,520 --> 00:04:31,360
may be held responsible
for the use

83
00:04:31,440 --> 00:04:34,240
which may be made of
the information contained therein.

84
00:04:34,800 --> 00:04:36,800
This was an ADLAB PRO video.

85
00:04:36,880 --> 00:04:40,520
ADLAB PRO partners are:
University of Trieste,

86
00:04:40,600 --> 00:04:43,600
Universitat Autònoma
de Barcelona,

87
00:04:43,680 --> 00:04:47,680
University of Antwerp,
Adam Mickiewicz University,

88
00:04:47,760 --> 00:04:50,760
RTV Slovenia,
Utopian Voices,

89
00:04:50,840 --> 00:04:55,040
Soundfocus and Royal National
Institute of Blind People.

90
00:04:55,120 --> 00:04:58,080
ADLAB PRO was funded
by the European Union.

91
00:04:58,160 --> 00:05:01,920
This video was made available to you
on a Creative Commons licence.