Results of PACTE Group’s Experimental Research on Translation Competence Acquisition. The Acquisition of the Instrumental Sub-Competence

During the second decade of the twenty-first century, documentation in electronic format has come to form a normal part of the workplace for all professional translators. The aim of this article is to present the results of the acquisition of the instrumental subcompetence, which is based on the use of electronic resources. These results are part of empirical-experimental research carried out by the PACTE group on Translation Competence Acquisition. In this study, the evolution of the acquisition of this sub-competence for five groups of translation students, from the first year of their degree course to their entry into the labour market, was measured using a methodological design that simulates a longitudinal study. The experiment was carried out in 2011 with 130 students on the Translation and Interpreting degree course. Five indicators related to the direct and inverse translation processes are analysed: number of resources, time taken on searches, time taken on searches at each stage, number and variety of searches. These indicators are then correlated with the quality of the final product of the translation process: translation acceptability. The results produced by the translation students are compared with those obtained in the Translation Competence experiment, carried out by the PACTE group in 2005-2006 with 35 professional translators i .


Introduction
In this article we present the partial results of an experiment carried out by the PACTE group on Translation Competence Acquisition (TCA) ii .Specifically, it includes the results corresponding to the Use of Instrumental Resources variable (UIR), which is defined as the documentation strategies used when consulting resources in electronic format.This variable is linked to the instrumental sub-competence in PACTE's Translation Competence model, and it is defined as "predominantly procedural knowledge related to the use of documentation sources and information and communication technologies applied to translation: dictionaries of all kinds, encyclopaedias, grammar books, style guides, parallel texts, electronic corpora, search engines, etc." (PACTE 2003:59).
We describe how translation students use electronic resources during the translation process and we focus on the gradual evolution of the characteristics of this use during their regular, higher education time.The experiment dealt with five groups, the last of which was made up of recent graduates.The methodology used for collecting the data was the simulation of a longitudinal study by taking simultaneous measurements from groups of first-year, second-year, third-year, and fourth-year students, as well as from a group of recent graduates, a total of 130 subjects, instead of taking repeated measurements from one sample of students over a period of five years (PACTE 2014:96-97).
We also give the results obtained for the same variable (UIR) for the group of translators (35 subjects) from the PACTE's Translation Competence (TC) experiment.
The paper begins with a brief contextualisation of the present study within the setting of other studies on documentation strategies in a digital environment.This is followed by the most relevant information on the research design: design elements common to TC and TCA experiments and those which are specific to the TCA study.The main part is devoted to the results obtained for the indicators of the UIR variable.At the end, we analyse the relation between the indicators proper of the documentation strategies during the translation process and the acceptability indicator, which informs about the quality of the translation product.Finally, we formulate our concluding remarks.

Previous studies on translation and the use of documentation strategies in electronic environment
The impact of documentation resources in electronic and online format has increased considerably since computers and the internet have become essential [page 21 in published version:] tools for the work of professional translators.During the training of translators both the quantity and the diversity of documentation resources has to be considered in order to provide students with an overview of these resources so that they can adopt them and use the documentation tools presented in the classroom, but also develop and perfect their own documentation strategies and discover new ways of finding information iii .
The use of documentation resources in electronic and online format has been a qualitative leap for translators who have been practicing in the profession for a long time because of the variety of resources and speed of searches.However, for the new generations who have grown up with rapid-access ICT from a wide range of tools and computer devices, the revolution of new pathways to documentation seems attenuated.
Despite being accustomed to using documentation resources and computer tools, electronic and online documentation also raised a series of difficulties which were often caused by the very broad range of options for finding the required information: what to search for (define the required information); how to search (define the strategy for finding the required information); where to search (select the sources of documentation in terms of reliability); and how much to search for (plan efficient searches in terms of the time spent searching balanced with the quality of the information found).
From the perspective of research into translation teaching, Olalla-Soler and Vert Bolaños (2013) used a questionnaire of the Spanish University Association of Trainee Translators and Interpreters to collect data about students and graduates from all Translation and Interpreting degree courses in 18 Spanish universities to see how they perceive their competence in the use of tools and documentation resources related to translation as a professional activity.Both for the documentation resources and the computer tools, a rise in the evaluation of efficacy of use was observed (Olalla-Soler & Vert Bolaños 2013:351-352).A particularly big leap was noted from first and second year to third and fourth year.This is consistent with the fact that most Spanish universities offer subjects in computer technology applied to translation in the first or second year (2013:340).
From the perspective of research into professional translation practice there has been growing interest among researchers looking at issues of usability of documentation resources and tools, in conjunction with research on effectiveness of these artefacts.For example, Bergenholtz and Gouws (2010) carried out a study of access to information from a userdriven lexicographic approach.Kuznik (2012) describes the interactions between humans and artefacts to perform terminology searches as an important part of the work content in the translation companies.Risku (2014) gives an insight into the [page 22 in published version:] interactions of the translator with the source of information in the workplace during the completion of a translation task, from the perspectives of situated, embodied, and extended cognition.Massey and Ehrensberger-Dow (2011) identify educational and ergonomic needs through the study of technical and instrumental competence in the translator's workplace.Cid and Perpinyà (2013) underline the importance of documentation in specialised translation and Muñoz-Miquel (2014) demonstrates its importance in the case of specialised medical translation in Spain.
All this means that translation today is a paradigmatic universal, post-industrial, knowledge-based and innovative service (Kuznik 2014) and that documentation activities constitute a very important part of the content of in-house (Kuznik 2012) and freelance (Risku 2014) translators' work.

Conclusions from the PACTE's Translation Competence experiment regarding the Use of Instrumental Resources variable
We will now present the most significant conclusions of the UIR variable in PACTE's TC experiment (Hurtado Albir 2017; Kuznik 2017), which served as a starting point for designing the UIR variable in PACTE's TCA experiment and for comparing the results obtained from the TC sample of professional translators and from the TCA sample of trainee translators.In the TC experiment, a group of translators and a group of language teachers were compared in terms of the time spent on searches during the translation task, the variety of types of resources and the variety of searches used during the translation task.For more details regarding the methodological design of the TC experiment, see 3. The group of translators spent more time on searches than the group of foreign language teachers in both direct and inverse translation.In both groups, the longest time taken was at the development and revision stages.
The group of translators used many more different types of resources than the group of teachers, both in direct and in inverse translation.The number of searches the translators carried out in direct and inverse translation was much larger than the number of searches carried out by the teachers.
Characteristic of the group of translators was their use of a wide variety of searches.Analysis of the relationship between the indicators variety of searches and acceptability showed that the greater the variety of searches, the better the results in both experiment groups in inverse translation.

[page 23 in published version:]
The results of the study have demonstrated the importance and use of electronic resources as characteristic of TC.In comparison with the group of teachers, translators spent more time on searches, used a larger number of resources, carried out a larger number of searches, and used different types of searches combined in different sequences.This shows efficient use of instrumental resources which in turn produces more acceptable translations.
The results obtained in the TC experiment served as a point of departure for the improvement of the methodological design of the TCA experiment.The adequacy of the indicators of the variable UIR and their measurement capability were assessed (see 3.1.3.The variable Use of Instrumental Resources.Definition and indicators).Moreover, these results served as a point of comparison between the translation competence acquisition process in translator training and the translation competence of professional translators, who served as the control group in the TCA experiment.

PACTE's Translation Competence and Acquisition of Translation Competence experiments
The overall goal of PACTE's research is to study Translation Competence Acquisition.In order to reach this goal, PACTE broke the work down into two stages: (1) the Translation Competence (TC) experiment, and (2) the Translation Competence Acquisition experiment(TCA).The TCA studies included a pilot test (June 2011) and the experiment itself (November 2011).We define TCA as follows: 1.A dynamic, spiral process, which, like all learning processes, evolves from novice knowledge (pre-translation competence) to expert knowledge (translation competence); it requires learning competence (learning strategies).During the process, both declarative and procedural types of knowledge are integrated, developed, and restructured.2. A process in which the development of procedural knowledge-and, consequently, of the strategic sub-competence-is essential.3. A process in which the TC sub-competences are developed and restructured.(PACTE 2014:92-93) The TCA research uses the same TC variables, indicators and instruments validated in the TC research, although some instruments have been adapted (PACTE 2014:87).In the following section we present the design elements common to both stages followed by specific design elements of the TCA [page 24 in published version:] experiment (hypotheses, population and sample, data collection and data analysis).

Languages and tasks of the experiment
The language combinations used in the experiment were: English, German or French (L2) to Spanish or Catalan (L1) for direct translations and Spanish (L1) to English, German or French (L2) for inverse translations.
The experiment tasks to be carried out by the subjects were as follows: 1. Completion of a questionnaire on knowledge of translation.2. Direct translation: translation of a text into L1, with computers connected to internet but without computer-assisted translation tools, followed by completion of a questionnaire on the translation problems involved.3. Inverse translation: translation of a text into L2, with computers connected to internet but without computer-assisted translation tools, followed by completion of a questionnaire on the translation problems involved.

Texts selected and rich points
The texts selected for use in the experiment were trialled in a pilot study carried out in 2004 (PACTE 2005a(PACTE , 2005b)).For the purposes of direct translation, parallel texts in English, German and French on the subject of computer viruses were used: "E-mail virus strikes in new form" (The Guardian, 6 June 2003), "Wurm in der Leitung" (Frankfurter Allgemeine Zeitung, 14 June 2003) and "Bugbear.B, le virus informatique qui lit par-dessus l'épaule de ses victimes" (Le Monde, 13 June 2003).The text used for inverse translation, a tourist brochure, was the same for all subjects in each group.It was translated into English, German or French.
Since PACTE considers translation to be a problem-solving process, data was gathered from specific source-text segments that contained prototypical translation problems.These we refer to as rich points.The rich points (Table 1)

The variable Use of Instrumental Resources. Definition and indicators
The UIR variable is one of the six dependent variables established by PACTE with the aim of measuring the TC and its acquisition, beside such variables as (1) knowledge of translation; (2) translation project; (3) identification and solution of translation problems; (4) decisionmaking; and (5) efficacy of the translation process (PACTE 2014:95).The UIR variable is related to the instrumental sub-competence.The objective of this variable is to collect data on the use of resources in electronic format during the translation task, recorded on-screen using Camtasia software.This variable is defined as the documentation strategies used when consulting resources in electronic format: websites, dictionaries and encyclopaedias in CD-ROM, etc. (Table 2).

[page 26 in published version:]
The UIR variable provides complementary data on the use of external support during the decision-making process (variable: Decision-making; PACTE 2009) and on the time taken on documentation strategies using electronic resources included in the time of completion of the entire translation task (variable: Efficacy of the translation process).
Data for five indicators of UIR variable are gathered during the translation process (number of resources, total time taken on searches, time taken on searches at each stage, number of searches, and variety of searches) and one indicator refers to the quality of the translation product (acceptability).

Research hypotheses
The general hypotheses for the TCA study include the following statements (PACTE 2014:94): 1. TC comprises several inter-related sub-competences.
2. The development of the strategic, instrumental, and knowledge-of-translation subcompetences is particularly important.3.Not all sub-competences develop in parallel, i.e. at the same time and at the same rate.4. Learning strategies must also be acquired.5. TCA is dependent upon directionality (direct/inverse translation), language pairs in use, the field of specialised translation (legal, literary translation, etc.) and the learning environment.
The empirical hypothesis for the UIR variable states that there is a relationship between the degree of TC and the use of external resources.The operational hypotheses for the TCA study include the following statements: 1. Differences can be observed at each level of TC acquisition in relation to the number of resources consulted.2. Differences can be observed at each level of TC acquisition in relation to the time spent on searches.3. Differences can be observed at each level of TC acquisition in relation to the distribution in stages of the time spent on searches.4. Differences can be observed at each level of TC acquisition in relation to the number of searches made. 5. Differences can be observed at each level of TC acquisition in relation to the variety of searches made.
The research hypotheses that we state are not directly built upon the results obtained in the TC experiment, since our research is not focused on comparing translation students to professional translators, but rather on observing the translation competence acquisition process that takes places from first to the fourth-year students.Thus, we only compare fourthyear students' results to translators', since we do expect to observe that fourth-year students behave in a similar way as professional translators.Moreover, as mentioned in PACTE's definition of TCA (see 3. (PACTE forthcoming), Translation Competence Acquisition is a non-linear process and, therefore, our empirical hypotheses do not state a particular increase or decrease in the use of external resources as TCA progresses.

Universe and sample
The universe was that of trainee translators, and the sample was drawn from first to fourthyear undergraduate students and recent graduates from the Faculty of Translation and Interpreting, Universitat Autònoma de Barcelona.PACTE used an initial questionnaire to screen the experimental sample, selecting a group of approximately 30 students corresponding to each year from all those deemed suitable.
In total, 130 subjects divided into five groups took part in the experiment which was carried out in November 2011 (Table 3).

[page 29 in published version:]
For the UIR variable, of the 130 subjects that made up the sample, data for 118 subjects were analysed for direct translation and 125 for inverse translation.The cause of the difference between the total sample and the number of subjects from whom data was collected is that some subjects' recordings were lost and some subjects did not have Spanish or Catalan as mother tongue and therefore they were not taken into account in the analysis for this variable.For the transversal indicator of acceptability 128 valid cases of direct and inverse translation were taken into account.Two subjects for each of the directions were excluded from the analysis since they had neither Spanish nor Catalan as their mother tongue.The students spoke Spanish or Catalan as their mother tongue, were within the average age group for their year, had not transferred from another degree course and had passed at least 80% of the previous year's subjects.

The data collection instrument and procedure. Translation process recordings
The tool used to record the translation process for this variable was the Camtasia software, which generated .avifiles.On-screen recordings were made of all subjects' actions during the translation process.We used Camtasia because, by being able to record all the actions carried out by the subject using a computer, we could obtain data related to: (1) the time spent on the translation and in each stage of the translation; (2) all the actions that the subject carried out during the translation, such as reformulations, pauses or deletions, and (3) the documentation strategies and the use of resources by the subject.Additionally, this software did not interfere with the translation process of the subject, thereby complying with the principle of ecological validity (Neunzig 2000).
The .avi recordings were later viewed and data was recorded in a database (Figure 1).Recorded data included:  Information about the subjects (code, year and L2). Information about the rich point consulted and the time of the consultation (the rich point referred to in the consultation, the stage at which it was made, start time and end time of the consultation). Information about resources and consultations (resource name, resource type and type of consultation).
For each consultation of each rich point by each subject a register was added in the database.In total we registered 1,425 consultations for direct translations and 2,034 for inverse translations.
[page 30 in published version:] Figure 1.The database created to register information on the Use of instrumental resourcesvariable

Quantitative analysis of the data collected
Although the rich points are multidimensional (some are more complex than others, some combine different categories), for the quantitative analysis they were considered to be of equal weight.We calculated one datum per subject (average or sum depending on the case) and then an average for the year.
All the results generated tables and graphs appropriate to the type of indicator (categorical or quantitative) with the basic descriptive statistics.To analyse the relationship between the indicators of the UIR variable and the transversal indicator acceptability, correlations for each year were calculated.

Training received by students of the Translation and Interpreting degree course
In the Faculty of Translation and Interpreting, Universitat Autònoma de Barcelona, the first-, second-and third-year students were taking the new, EHEA-adapted Translation and Interpreting degree course--a competence-based training programme that places great emphasis on progression in the acquisition of competences.Fourth-year students were taking the older, un adapted Translation and Interpreting degree course, the same programme the recent graduates had studied; pedagogical input and progression for these students was different.
[page 31 in published version:] Second-year students had taken single-semester courses in Introduction to Translation and Introduction to Translation Technologies.Third-year students had also taken a singlesemester course on Documentation Applied to Translation and a two-semester subject on Direct Translation.Fourth-and fifth-year students, as well as graduates, had taken, in addition, single-semester courses such as Translation Theory, Specialised Direct Translation, Terminology, Translation into L2, and another course on Translation Technologies.All of them had taken courses geared to developing their competence in their L2 and L1.

Evolution of the acquisition of the Use of Instrumental Resources. Measuring of indicators and results obtained
In the first part of this section, we present the measurement methods and results for the following five indicators: number of resources, total time taken on searches, time taken on searches at each stage, and number of searches and variety of searches.

Measurement
In the TC experiment, a list of the electronic resources consulted by subjects was drawn up after viewing all Camtasia recordings (Kuznik 2017), and in the TCA experiment we use the same list as in the TC experiment.This list included the following eight items: 1. Search engines (normally Google).
[page 32 in published version:] 7. Online corpora or corpora in CD-ROM format (RAE, Collins, Google used as a corpus, etc.).8. Specialised or field-specific portals (news items, tourist information, information related to the subject of the texts to be translated).
Subsequently the number of types of the electronic resources consulted to resolve rich points in direct and inverse translation was calculated.

Results
The results for the number of types of resources in direct and inverse translations are presented in Table 4.In direct translation, the evolution of the number of resource types consulted during a translation task, between the first year students and graduates, does not present a completely regular pattern.It is true that first-year students use the least number of resource types (2.62) and graduates use the greatest number of types of resources (3.72) but between these two groups there is a downward trend: after a leap from the first to the second year (2.62 and 3.71, respectively), trainee translators use fewer and fewer types of resources; in other words, the number gradually decreases from the second to the fourth year.
In inverse translation, the pattern of the number of types of resources used presents a more regular, progressive shape: it increases constantly from the first year (2.96) to the graduates (4.62), with an exception for the second year when this number is very high (4.17)compared to first year students (2.96).This leap between first and second year both in direct and inverse translation is caused by the lack of translation experience, so students are not familiar with a wider range of resources.

[page 33 in published version:]
In direct translation, the number of resources decreases from third year because of the use of internal support; that is, the students' linguistic and extra-linguistic knowledge, the knowledge of translation and cognitive strategies to solve translation problems.However, this pattern is not observable in inverse translation, probably because internal support by itself is not enough to solve translation problems into L2 and students need to rely on consultations iv .
Generally, a wider variety of resources is used in inverse translation than in direct translation.
Although in the TC experiment a slightly different calculation procedure for this indicator was applied v , it can be observed that translators use more resources than students in direct and inverse translation.

Measurement
The time taken on searches is measured as follows: first, for each student the time invested in consultations to find solutions to the five rich points in direct and five rich points in the inverse translation is calculated in minutes, from the start time and end time of the Taking the sum of these times per subject the mean value was calculated for each of the five groups of translator trainees.

Results
The results for the time taken on searches in direct and inverse translation are presented in Table 5.
[page 34 in published version:] In direct translation, the time taken on searches tends to fall from first (9.52) to fourth year (8.36).However, second-year students score the highest mean of time taken on searches (11.25).Although a decreasing trend is observable, graduates (9.72) spend more time on searches than any other group except second-year students.
In inverse translation, there is a decreasing trend from second-to fourth-year students, although fourth-year students spend a similar time on searches to first-year students (10.48).Second-year students register the highest value (13.17) and graduates (13.05) spend more time on searches than the rest of the groups.The trend for both direct and inverse translation is similar, although values are higher for inverse translation.This is the case because the efficacy of searches increases and internal support is used to solve translation problems.
Generally, the time spent on searches is higher in inverse translation, probably because external support for solving problems in the L2 is more frequent in inverse translation than in direct translation.

[page 35 in published version:]
When comparing translators to fourth-year students and graduates, it is observable that in direct translation these two groups of students spend a similar time on searches as translators do.However, in inverse translation fourth-year students and graduates spend more time on searches than translators do.This indicator provides information on the time subjects spent on searches at each stage (orientation, development, revision) of the translation process to find solutions to the rich points in direct and inverse translation.It is complementary data to the previous indicator.We can also see how subjects distribute the time spent on searches throughout the whole translation process; in other words, whether they prefer to do these searches at the beginning of the translation task, during the translation task or during the revision stage.
Six data (in minutes and percentages) were obtained for each subject: three for direct and three for inverse translation.Then the mean values for each group were calculated.
Given that the lowest value for the measurement of indicators related to consultation times (time taken on searches and time taken on searches at each stage) was 1 minute, the sum of the times for the different stages varies slightly in some cases from the time taken on searches, presented above in Table 5 (the cases of overlap in times for the stages in direct translation: graduates; and in inverse translation: second-and third-year students).

Results
In Table 6 we include the results as percentages for the time taken on searches at the orientation, development and revision stages in direct translation.
[page 36 in published version:] In Table 7 we include the results in percentage for the time taken on searches at the orientation, development and revision stages in inverse translation.Observing the behaviour of all groups we can confirm that most of the consultations are concentrated in the development stage, with that concentration being more pronounced for inverse translation than direct translation.Revision is the second highest stage for consultations but, in this case, for inverse translations consultations are shorter than for direct translations, probably because students do not find consultations as useful as in direct translation since they do not have the same criteria to evaluate and compare the solutions found in their L2 as they do in their L1, so revision searches are limited.There are very few consultations in the orientation stage.
Looking at the results by group we see very different behaviour by the second-year students.In direct translation these students spend much more time than the other groups on consultations in the orientation stage (10.9%), the greatest time of all in the development stage (84.4%) and surprisingly little time in the revision stage (4.7%).These results would seem to show that in direct translation they carry out their consultations at the beginning of the translation process.In inverse translation the consultations are concentrated clearly in the development stage (94.6%) and very little in the revision stage (6.3%).Generally, secondyear students avoid carrying out consultations at the end of the translation process and this behaviour differentiates them from the rest of the sample.These students have already taken one semester of translation practice and so they are more aware of translation problems than the first years but they still do not have the resources for solving them that the third-and fourth-year students and graduates have.
Student behaviour for all years, including in the final stage of training is very different from that of the professionals, especially in direct translation: students carry out far fewer consultations in the orientation stage, many more in the development stage and far fewer in the revision stage.

Measurement
The number of searches carried out by subjects to find solutions to the total of ten rich points, five in direct and five in inverse translation, was used to measure this indicator.

Results
The results for the number of searches in direct and inverse translation are presented in Table 8.In direct translation, a decreasing tendency is observed from first-(mean 11.95) to fourth-year (10.36), except in second-year students, who score the highest value (14.96).This tendency is due to the more frequent use of internal support to solve translation problems.Fourth-year students perform the fewest number of searches (10.36).Graduates (12.61) perform almost as many searches as first-year students (11.95).When comparing fourth-year students and graduates to translators (20.51), it is observed that translators perform more searches.
In inverse translation, there is a notable leap between first-(11.83)and second-year students (17.71).There is an increasing trend from first-to third-year students (18.54), although the number of searches decreases in fourth-year students (14.69).Graduates register a higher value than any other group of trainee translators (18.67).When comparing fourthyear students and graduates to translators, it is observed that translators perform more searches (28.8).
In general, more searches are performed in inverse translation than in direct translation, probably since students cannot rely on their internal support only to solve translation problems into L2.
Translator trainees carry out far fewer searches than translators in the TC experiment.
[page 39 in published version:]

Variety of searches
This indicator was designed to measure the variety of searches used by subjects to find solutions to the selected rich points, i.e. five in direct translation and five in inverse translation.The variety of searches was measured in two steps, as described below.

Measurement
We used the same list of the types of searches as for the TC experiment (Kuznik 2017).This list included the following thirteen items: 1. Searches using keywords.Searches were carried out using a search engine, no matter how many words were introduced at one time, whether they were separated by a space or using Boolean connectors (+/-; AND, OR, AND NOT, etc.).2. Searches for equivalents.Equivalence in terms was sought using resources in both L1 and L2 languages.3. Searches for definitions.Definitions of terms were sought using resources in only one language.4. Exact searches.Inverted commas were used to delimit items.5. Searches for synonyms.Synonyms for terms were sought using resources in the same language.6. Searches in context.Electronic corpora or a search engine was used as a corpus to determine the use or frequency of use of expressions in context.7. Searches in encyclopaedias for clarification.Wikipedia was usually used for clarification of polysemy.8. Searches in websites.The option CTRL+F was used to search for information on a website.9. Cache searches.A cache (copy of the web page the search engine saw when it visited the URL) was used to obtain information.10.Searches using preferred region.The option "Google Advanced Searchnarrow your results by region" was used.11.Searches using preferred language.The option "Google Advanced Searchnarrow your results by language" was used.12. Searches using preferred date.The option "Google Advanced Searchnarrow your results by last update" was used.[page 40 in published version:] 13.Searches followed by correction.A solution provided by the search engine ("Search instead for...") was accepted and used to correct the outcome of the initial search.
Subsequently the number of types of searches used to resolve rich points in direct and inverse translation was calculated.

Results
The results for the variety of searches (step one: types of searches) in direct and inverse translation are presented in Table 9.In direct translation, there is a notable leap between first-(2.43)and second-year students (3.29) due to the contact with translation practice.There is no clear tendency, since the variety of searches increases up to second-year students and then decreases until fourthyear students (2.75) since internal support increases.Second-year students score the highest value, followed by graduates (3.11).The results for the graduates are quite similar to the translators' variety of searches (3.09).

[page 41 in published version:]
In inverse translation, there is an increasing trend from first-(2.48) to fourth-year students (3.59).There is a notable leap between first-and second-year students (3.5).Graduates register the widest variety of searches (4.0).The mean value for the translators is lower (3.20).
In general, there is a wider variety of searches in inverse translation than in direct translation.

Measurement
The 13 types of searches were then grouped into five categories according to the combination of resources used to resolve the rich points selected.The categories established were: 1. None: no search of any kind was performed.2. Simple: only one type of search was performed.3. Double: a combination of two types of search was performed.4. Combined: a combination of three, four or five types of search was performed.5. Multiple: a combination of more than five types of search was performed.
In this second step, 10 data per subject were obtained--five for direct and five for inverse translation.These data represented the frequency of use of each type of search strategy as reflected in the five different categories.

Results
The results for the variety of searches (step two: combinations of types of searches) in direct and inverse translation are presented in Table 10.In direct and inverse translation, there is a general decreasing trend between the firstyear students and the graduates in all combinations (except for combined use in direct and inverse translation and multiple use in inverse translation), probably since they use internal support more frequently to solve translation problems.The most frequently used combinations of searches in both direct and inverse translation in the group of translators are double (34.3%) and combined searches (40%).Compared to fourth-year students and graduates, these two groups of students use combined searches much more frequently (60.7% in fourth-year students and 77.8% in graduates).However, they use [page 42 in published version:] double searches less frequently than translators do even if double search is the second most frequently used combination (17.9% in fourth-year students and 16.7% in graduates).
Analysing the behaviour of the different groups, the first-year students seem to show a clearly different form of working than the others, both in direct and inverse translation.These students prefer double use (42.9%) and often single use (14.3%) but make very little use of combined consultations (38.1%) in comparison with the other groups.For inverse translation this group shows similar behaviour: there is a clear preference for double use (47.8%), combined use (39.1%) and single use (13.0%), in that order and they do not carry out any multiple consultations.In general, they tend to combine far fewer types of electronic consultation resources than the other groups because they still lack experience in translation and the use of instrumental resources.
[page 43 in published version:]

Acceptability. Relation between the documentation strategies during the translation process and the quality of the translation product
In this section, we include the results for the relation between the indicators proper of the documentation strategies during the translation process and the acceptability indicator in the TCA experiment.

20
Acceptability is related to the quality of the translation product.It is the only indicator that is used in conjunction with the specific indicators of each variable.In the PACTE research project, acceptability is defined in terms of whether or not the solution effectively communicates (a) the meaning of the source text; (b) the function of the translation (within the context of the translation brief, the readers' expectations, genre conventions in the target culture); and c) makes use of appropriate language.
Three degrees of acceptability were established and defined (PACTE 2007a(PACTE , 2007b(PACTE , 2008)): -Acceptable solution (A): the solution activates all the relevant connotations of the ST in the TT as regards the meaning of the ST, function of the translation and language use.-Semi-Acceptable solution (SA): the solution activates some of the relevant connotations of the ST in the TT, and maintains the coherence of the TT with regard to the meaning of the ST, function of the translation and language use.-Non-Acceptable solution (NA): the solution activates none of the relevant connotations of the ST in the TT, or introduces connotations that are incoherent with regard to the meaning of the ST, function of the translation and/or language use.The criteria used to evaluate the acceptability of the solutions obtained for the translation of rich points were based on the description of each; the type of translation problem represented; and possible solutions that members of PACTE Group had established a priori as Acceptable, Semi-Acceptable or Non-Acceptable solutions(PACTE in press).
To ensure validity in the application of the criteria established in all language combinations, a peer review was established for each rich point in each language combination.Reviewers were translation teachers with experience in translation, some whom in each language combination were [page 44 in published version:] members of the target reader group.Where doubts arose, members of the research group as a whole were consulted.
When the results obtained for each of the three factors defining Acceptability (meaning, function and language use) were triangulated, 27 different permutations were defined and classified into three levels of Acceptability (A, SA, NA) giving priority to the effective transmission of the meaning of the source text.Once each permutation had been assigned to one of the three categories (A, SA, NA) they were redistributed in three levels in accordance with their overall level of Acceptability.Subsequently, each category was given a numeric value: A = 1; SA = 0.5; NA = 0.

Descriptive analysis
Figure 2 shows the descriptive results for acceptability in the TCA experiment.vii This is a post-print version of the following published paper: Kuznik, Anna;Olalla-Soler, Christian (2018)   In inverse translation, there is an evident leap between the third (0.39) and the fourth year (0.51) which is due to the fact that students have received [page 45 in published version:] pedagogical input (classes in inverse translation).Acceptability for graduates (0.49) falls slightly compared with fourth-year students (0.51).This is probably due to graduates' contact with the labour market where the quality of translations is tested and qualified by the clients.
Qualitative analysis of the relationship between Acceptability of the translations (Figure 2) and the three indicators of the UIR variable: number of resources (see Table 4), time taken on searches (see Table 5) and number of searches (see Table 8) in TCA, does not seem to show any clear tendency or evidence for the evolution of this relationship.
However, we also carried out a statistical analysis to find possible relationships between Acceptability and these three indicators of the UIR variable.

Statistical analysis
Quantitative statistical analysis of the degree of association among the indicators in each separate group using the Spearman correlation gave the following results, presented in Table 11.
This is a post-print version of the following published paper: Kuznik, Anna;Olalla-Soler, Christian (2018)  In both direct and inverse translation, there are no relevant positive or negative correlations between the UIR indicators and acceptability.This means that spending more or less time on searches, using a higher or lower number of [page 46 in published version:] resources and performing a higher or lower number of searches do not correlate with the quality of the solutions.
When comparing students to translators in direct translation, it can be observed that both groups perform similarly (no relationship) except for the indicator time taken on searches.The more time translators spend on searches, the worse the solutions for rich points are.However, students do not present any clear pattern in this indicator.
When comparing students to translators in inverse translation, it can be observed that both groups perform differently.All relationships are positive in the case of translators (the greater the value for these indicators, the higher the acceptability), but there is no observable pattern towards a positive or negative relationship in students.

Conclusions
As demonstrated throughout this article, the analysis was carried out using a complex and laborious methodology, handling a large quantity of data due to the number of subjects in the experiment, the quantity and complexity of the indicators and the separation of direct and inverse translations.To reduce the effort required for this research we followed the principle of research economy and only considered data from five rich points in each text.
The analysis carried out was mainly quantitative and so some of the differences which almost certainly exist between the subjects and their individual way of using electronic and online resources were not observed.The quantitative categories of analysis (8 types of resources and 13 types of searches) were also identified for the purpose of quantitative analysis but they should perhaps be redefined and contextualised for qualitative analysis.
From analysis of the data from the TCA experiment on documentation strategies used when consulting resources in electronic format we have reached the following conclusions: 1. Homogeneous collective.Students from all the years together form a fairly homogeneous collective in terms of their behaviour and use of documentation strategies when consulting resources in electronic format.2. Non-linear progression.The evolution of the acquisition of documentation strategies used when consulting resources in electronic format overall has a similar profile for all the indicators of this variable: the evolution is shown by a gentle, wave-like line, starting at its lowest point with the first-years, moving up to the second year, and then falling [page 47 in published version] slightly for the third-and fourth-years before finally rising again for graduates.3. Differences between first-and second-year students and specific behaviour of both groups.In terms of the profile of evolution the following elements of behaviour in the first-and second-year groups are significant:  there is a clear quantitative leap between first-year and second-year students in the indicators studied: the number of resource types used in inverse translation and the number of searches in inverse translation;  very different behavioural traits emerge: i. first-year students tend to combine electronic consultation resources types much less than all the other groups; ii.second-year students concentrate their consultations on the orientation stage of direct translation and in the development stage of inverse translation, avoiding consultations at the end of the translation process, which is very different behaviour from the other groups in the sample.First-year students do not have experience in translation and still are not aware of the possibilities offered by the different instrumental resource types.Second-year students, having undertaken training in translation practice during the first semester, are more aware of the problems of translation and know about more sources of electronic documentation; consequently, they take greater advantage of these sources during the orientation and development stages, although they have not yet fully developed a concern for the quality of their work which leads them to spend little time on the revision stage.
4. Intensification of the use of instrumental resources in inverse translation.Inverse translation also involves a more intensive use of electronic resources for all the indicators studied.
This situation is probably due to the fact that since the students do not have access to as many resources in their L2, they need to use more instrumental resources to bridge that gap.
5. Lack of association with acceptability.The correlation coefficients show that spending more or less time on searches, using a higher or lower number of resources and performing a higher or lower number of searches does not correlate with the quality of the solutions.Nor are significant differences observed in this respect between direct and inverse translation.This implies that the use of instrumental resources depends on the specific needs of the translation student in order to solve a translation problem caused by the lack of internal support (linguistic [page 48 in published version:] and extra-linguistic knowledge, knowledge of translation and cognitive strategies) in an acceptable way.
These results in both direct and inverse translation show that the use of instrumental resources is not linked to the quality of the solutions given.Some students give acceptable solutions with a low number of resources, little time taken on searches and a low number of This is a post-print version of the following published paper: Kuznik, Anna;Olalla-Soler, Christian (2018)

24
searches while other students need many resources, many searches and a lot of time in order to achieve an acceptable solution.This indicates that instrumental resources are used to complement internal support, that is, the students' linguistic and extra-linguistic knowledge, their knowledge of translation and cognitive strategies to solve translation problems.The use of this complement is therefore linked to personal needs and difficulties, and it is not possible to conclude whether the higher or lower use of instrumental resources leads to better results.
From a comparison of the results of the TCA experiment with the translators of the TC we can conclude the following: 1.In the case of TCA, translation graduates are placed below the results obtained for professional translators in the following cases:  Number of resources  Time taken on searches in the orientation and revision stages  Number of searches  Double use (two types of search) and multiple use of search types (more than five) The trainee translators' lack of translation experience distances their behaviour greatly from that of the professional translators.
2. In the case of TCA, translation graduates are placed above the professional translators in the following cases:  Time taken on searches  Time taken on searches in the development stage  Variety of searches (search types) (a slightly higher value)  Combined searches (three types) The time spent on searches and the concentration of their searches in the development stage shows the trainee translators' need for more time to carry out their searches compared with the professional translators, especially at the beginning of the development stage of the translation task.
As regards the variety of searches, trainee translators have a clear preference for three different search types, but they still do not reach such a [page 49 in published version:] high level of acceptability as the professional translators.They are not as effective as the professionals since they need to carry out more searches.
3. In the case of TCA, no significant relationship was found between acceptability and the three indicators of the UIR variable: number of resources, time taken on searches, and number of searches.However, in the case of TC there was a relationship in inverse translations by professional translators: the greater the value in these indicators, the greater the acceptability.As explained above (conclusion 5) this lack of relationship in the case of students leads us to believe that the use of instrumental resources depends on the specific needs of the translation students.
As a final remark, we believe that it is important to stimulate the use of internal support in order to achieve a more efficient use of instrumental resources.This includes learning to take decisions without using external support exclusively and learning to evaluate documentation resources.It is also important to stimulate the efficient use of instrumental resources in translator training by learning to establish an order and priority of searches and by learning to evaluate their quality and the possibilities.
[page 50 in published version:] iv In the Decision-making variable it was observed that students used bilingual resources to a greater extent than professional translators.However, professional translators used predominantly internal support (all types of resources except bilingual resources).v Although in the TCA experiment the maximum value for this indicator was8, in the TC experiment the maximum value was higher than 8, because when a subject used the same type of resource several times, it was considered each time (not as 1, as it is in the TCA experiment).
vi Since there were time overlaps in some consultations at different stages, sometimes percentages amount to a higher value than 100.
vii In all the bar charts in this article, we include the information about the confidence interval in the upper section of each bar.In the TC experiment, the calculation procedure of the relationship between acceptability and the indicators of the Use of instrumental resources variable was performed qualitatively.
1. Translation Competence and Acquisition of Translation Competence experiments.Common elements of research design.
; values: 0 to 1; data per subject: 2(1 for direct and 1 for inverse translation).Whether or not the solution effectively communicates (a) the meaning of the source text; (b) the function of the translation (within the context of the translation brief, the readers' expectations, genre conventions in the target culture); and (c) makes use of appropriate language (; length of search (from start to finish); stage when search is carried out; number of searches; types of searches; acceptability[page 27 in published version:] PACTE's Translation Competence and Acquisition of Translation Competence experiments) and already proved in the transversal indicator Acceptability and in the variable Efficacy of the translation process [page 28 in published version:]

Figure 2 .
Figure 2. Acceptability of the translations in the TCA experiment viii [final note included in page 50 of published version:]

Table 1 .
Rich points included in the source texts in the PACTE's TC and TCA experiments

Table 2 .
Synoptic view of the variable Use of Instrumental Resources (Hurtado Albir 2017; Kuznik 2017)

Time Taken on Searches at each Stage (
This is a post-print version of the following published paper: Kuznik, Anna; Olalla-Soler, Christian (2018) "Results of PACTE Group's Experimental Research on Translation Competence Acquisition.The Acquisition of the Instrumental Sub-Competence", Across This is a post-print version of the following published paper: Kuznik, Anna; Olalla-Soler, Christian (2018) "Results of PACTE Group's Experimental Research on Translation Competence Acquisition.The Acquisition of the Instrumental Sub-Competence", Across

Table 3 .
Number of subjects analysed per group and directionality of translation, in the TCA experiment [included in the page 28 in published version]

Table 4 .
Number of resources (for the TCA experiment, standard deviation in brackets) This is a post-print version of the following published paper:Kuznik, Anna; Olalla-Soler, Christian (2018)"Results of PACTE Group's Experimental Research on Translation Competence Acquisition.The Acquisition of the Instrumental Sub-Competence", Across /akademiai.com/doi/abs/10.1556/084.2018.19.1.213 consultation, and registered in the database.For example, if a subject carried out a consultation at 15:30-15:31, and a second one at 15:35-15:35, it is calculated that this student has spent 3 minutes on consultations from 15:30 to 15:31 (2 minutes) and from 15:35 to 15:35 (1 minute).

Table 5 .
Time taken on searches, in minutes (for the TCA experiment, standard deviation in brackets)

Table 6 .
Time taken on searches at each stage in direct translation, mean percentage vi (for the TCA experiment,

Table 7 .
Time taken on searches at each stage in inverse translation, mean percentage (for the TCA experiment, This is a post-print version of the following published paper: Kuznik, Anna; Olalla-Soler, Christian (2018) "Results of PACTE Group's Experimental Research on Translation Competence Acquisition.The Acquisition of the Instrumental Sub-Competence", Across

Table 8 .
Number of searches (for the TCA experiment, standard deviation in brackets)

Table 9 .
Variety of searches; step one: types of searches (for the TCA experiment, standard deviation in

Table 10 .
Variety of searches; step two: combinations of types of searches [included in page 42 in published This is a post-print version of the following published paper: Kuznik, Anna; Olalla-Soler, Christian (2018) "Results of PACTE Group's Experimental Research on Translation Competence Acquisition.The Acquisition of the Instrumental Sub-Competence", Across "Results of PACTE Group's Experimental Research on Translation Competence Acquisition.The Acquisition of the Instrumental Sub-Competence", Across /akademiai.com/doi/abs/10.1556/084.2018.19.1.221

Table 11 .
"Results of PACTE Group's Experimental Research on Translation Competence Acquisition.The Acquisition of the Instrumental Sub-Competence", Across Correlation between the Acceptability of the translation and the indicators of Use of Instrumental Resources (TCA experiment: Spearman correlation coefficient) "Results of PACTE Group's Experimental Research on Translation Competence Acquisition.The Acquisition of the Instrumental Sub-Competence", Across /akademiai.com/doi/abs/10.1556/084.2018.19.1.2