Annotation Tool | External Evaluation

– Step 1 of 3

PARTICIPANT

Please indicate your field of expertise in your own words: *

Do you have any previous text-annotation experience: *

No, this was my first time.
I’ve done it once or twice.
I do it occasionally.
I regularly engage in text annotation as part of my profession.

Which annotation tools have you used? *

Please indicate your technical knowledge: *

I can search information on the Internet.
I can write and format a document with MS Word.
I can identify the sources in a piece of news I am reading, or confirm that these are not present.
I can look up vocabulary that I don’t understand in online dictionaries or thesauri.
I can upload an image on a social network.

Check all that apply

Is English your mother tongue?

English competence *

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
My level of English was satisfactory for the needs of the task.	Strongly DisagreeMy level of English was satisfactory for the needs of the task. Strongly Disagree	DisagreeMy level of English was satisfactory for the needs of the task. Disagree	NeutralMy level of English was satisfactory for the needs of the task. Neutral	AgreeMy level of English was satisfactory for the needs of the task. Agree	Strongly AgreeMy level of English was satisfactory for the needs of the task. Strongly Agree

USABILITY OF THE ANNOTATION TOOL

Please indicate your agreement/disagreement with the following statements. *

	Strongly disagree	Somewhat disagree	Neither agree nor disagree	Somewhat agree	Strongly agree
I think I would like to use this annotation tool frequently	Strongly disagreeI think I would like to use this annotation tool frequently Strongly disagree	Somewhat disagreeI think I would like to use this annotation tool frequently Somewhat disagree	Neither agree nor disagreeI think I would like to use this annotation tool frequently Neither agree nor disagree	Somewhat agreeI think I would like to use this annotation tool frequently Somewhat agree	Strongly agreeI think I would like to use this annotation tool frequently Strongly agree
I found the tool unnecessarily complex	Strongly disagreeI found the tool unnecessarily complex Strongly disagree	Somewhat disagreeI found the tool unnecessarily complex Somewhat disagree	Neither agree nor disagreeI found the tool unnecessarily complex Neither agree nor disagree	Somewhat agreeI found the tool unnecessarily complex Somewhat agree	Strongly agreeI found the tool unnecessarily complex Strongly agree
I though the tool was easy to use	Strongly disagreeI though the tool was easy to use Strongly disagree	Somewhat disagreeI though the tool was easy to use Somewhat disagree	Neither agree nor disagreeI though the tool was easy to use Neither agree nor disagree	Somewhat agreeI though the tool was easy to use Somewhat agree	Strongly agreeI though the tool was easy to use Strongly agree
I think that I would need further technical support to be able to use the tool	Strongly disagreeI think that I would need further technical support to be able to use the tool Strongly disagree	Somewhat disagreeI think that I would need further technical support to be able to use the tool Somewhat disagree	Neither agree nor disagreeI think that I would need further technical support to be able to use the tool Neither agree nor disagree	Somewhat agreeI think that I would need further technical support to be able to use the tool Somewhat agree	Strongly agreeI think that I would need further technical support to be able to use the tool Strongly agree
I found the various functions of the tool well integrated	Strongly disagreeI found the various functions of the tool well integrated Strongly disagree	Somewhat disagreeI found the various functions of the tool well integrated Somewhat disagree	Neither agree nor disagreeI found the various functions of the tool well integrated Neither agree nor disagree	Somewhat agreeI found the various functions of the tool well integrated Somewhat agree	Strongly agreeI found the various functions of the tool well integrated Strongly agree
I thought there was too much inconsistency in this tool	Strongly disagreeI thought there was too much inconsistency in this tool Strongly disagree	Somewhat disagreeI thought there was too much inconsistency in this tool Somewhat disagree	Neither agree nor disagreeI thought there was too much inconsistency in this tool Neither agree nor disagree	Somewhat agreeI thought there was too much inconsistency in this tool Somewhat agree	Strongly agreeI thought there was too much inconsistency in this tool Strongly agree
I would imagine that most people would learn to use this tool very quickly	Strongly disagreeI would imagine that most people would learn to use this tool very quickly Strongly disagree	Somewhat disagreeI would imagine that most people would learn to use this tool very quickly Somewhat disagree	Neither agree nor disagreeI would imagine that most people would learn to use this tool very quickly Neither agree nor disagree	Somewhat agreeI would imagine that most people would learn to use this tool very quickly Somewhat agree	Strongly agreeI would imagine that most people would learn to use this tool very quickly Strongly agree
I found the tool very cumbersome to use	Strongly disagreeI found the tool very cumbersome to use Strongly disagree	Somewhat disagreeI found the tool very cumbersome to use Somewhat disagree	Neither agree nor disagreeI found the tool very cumbersome to use Neither agree nor disagree	Somewhat agreeI found the tool very cumbersome to use Somewhat agree	Strongly agreeI found the tool very cumbersome to use Strongly agree
I felt very confident using the tool	Strongly disagreeI felt very confident using the tool Strongly disagree	Somewhat disagreeI felt very confident using the tool Somewhat disagree	Neither agree nor disagreeI felt very confident using the tool Neither agree nor disagree	Somewhat agreeI felt very confident using the tool Somewhat agree	Strongly agreeI felt very confident using the tool Strongly agree
I needed to learn a lot of things before I could use this tool	Strongly disagreeI needed to learn a lot of things before I could use this tool Strongly disagree	Somewhat disagreeI needed to learn a lot of things before I could use this tool Somewhat disagree	Neither agree nor disagreeI needed to learn a lot of things before I could use this tool Neither agree nor disagree	Somewhat agreeI needed to learn a lot of things before I could use this tool Somewhat agree	Strongly agreeI needed to learn a lot of things before I could use this tool Strongly agree

List one or more aspects that positively affected your experience while using the VAST annotation tool: *

List one or more aspects that negatively affected your experience while using the tool: *

Do you have suggestions how to improve the overall usability of the tool? *

Do you have suggestions about possible functionalities that can be added to the tool? *

Did you use the radial organisation of labels?

No, I used only the label organisation as buttons
Yes, I used only the radial oranisation of labels.
I have used both organisations of labels.

OTHER INFORMATION

Were the instructions provided to you adequate for you to perform the annotation task? How could they be improved? *

Have you used any additional material (like the tool's documentation or shared video) to help you complete the task? If yes, do you have any suggestions for improvement? *

Is there any other information you would like to share with us? *

Cookie	Duration	Description
cookielawinfo-checkbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy	11 months	The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.