Annotation Tool Evaluation | Feb-23 – VAST Project Platform

Step 1 of 3

After using the VAST Annotation Tool, please respond to the following questionnaire. It contains 4 questions regarding your technical skills related to the task, 16 Likert-type questions and 7 open-ended questions. It should take about 10 minutes to complete it.

DEMOGRAPHICS

Please indicate your field of expertise in your own words: *

Do you have any previous text-annotation experience: *

No, this was my first time.
I’ve done it once or twice.
I do it occasionally.
I regularly engage in text annotation as part of my profession.

Please indicate your technical knowledge: *

I can search information on the Internet.
I can write and format a document with MS Word.
I can identify the sources in a piece of news I am reading, or confirm that these are not present.
I can look up vocabulary that I don’t understand in online dictionaries or thesauri.
I can upload an image on a social network.

Check all that apply

Is English your mother tongue?

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
My level of English was satisfactory for the needs of the task.	Strongly DisagreeMy level of English was satisfactory for the needs of the task. Strongly Disagree	DisagreeMy level of English was satisfactory for the needs of the task. Disagree	NeutralMy level of English was satisfactory for the needs of the task. Neutral	AgreeMy level of English was satisfactory for the needs of the task. Agree	Strongly AgreeMy level of English was satisfactory for the needs of the task. Strongly Agree

USABILITY OF THE ANNOTATION TOOL

Please indicate your agreement/disagreement with the following statements. *

	Strongly disagree	Somewhat disagree	Neither agree nor disagree	Somewhat agree	Strongly agree
I think I would like to use this annotation tool	Strongly disagreeI think I would like to use this annotation tool Strongly disagree	Somewhat disagreeI think I would like to use this annotation tool Somewhat disagree	Neither agree nor disagreeI think I would like to use this annotation tool Neither agree nor disagree	Somewhat agreeI think I would like to use this annotation tool Somewhat agree	Strongly agreeI think I would like to use this annotation tool Strongly agree
I found the tool unnecessarily complex	Strongly disagreeI found the tool unnecessarily complex Strongly disagree	Somewhat disagreeI found the tool unnecessarily complex Somewhat disagree	Neither agree nor disagreeI found the tool unnecessarily complex Neither agree nor disagree	Somewhat agreeI found the tool unnecessarily complex Somewhat agree	Strongly agreeI found the tool unnecessarily complex Strongly agree
I though the tool was easy to use	Strongly disagreeI though the tool was easy to use Strongly disagree	Somewhat disagreeI though the tool was easy to use Somewhat disagree	Neither agree nor disagreeI though the tool was easy to use Neither agree nor disagree	Somewhat agreeI though the tool was easy to use Somewhat agree	Strongly agreeI though the tool was easy to use Strongly agree
I think that I would need further technical support to be able to use the tool	Strongly disagreeI think that I would need further technical support to be able to use the tool Strongly disagree	Somewhat disagreeI think that I would need further technical support to be able to use the tool Somewhat disagree	Neither agree nor disagreeI think that I would need further technical support to be able to use the tool Neither agree nor disagree	Somewhat agreeI think that I would need further technical support to be able to use the tool Somewhat agree	Strongly agreeI think that I would need further technical support to be able to use the tool Strongly agree
I found the various functions of the tool well integrated	Strongly disagreeI found the various functions of the tool well integrated Strongly disagree	Somewhat disagreeI found the various functions of the tool well integrated Somewhat disagree	Neither agree nor disagreeI found the various functions of the tool well integrated Neither agree nor disagree	Somewhat agreeI found the various functions of the tool well integrated Somewhat agree	Strongly agreeI found the various functions of the tool well integrated Strongly agree
I thought there was too much inconsistency in this tool	Strongly disagreeI thought there was too much inconsistency in this tool Strongly disagree	Somewhat disagreeI thought there was too much inconsistency in this tool Somewhat disagree	Neither agree nor disagreeI thought there was too much inconsistency in this tool Neither agree nor disagree	Somewhat agreeI thought there was too much inconsistency in this tool Somewhat agree	Strongly agreeI thought there was too much inconsistency in this tool Strongly agree
I would imagine that most people would learn to use this tool very quickly	Strongly disagreeI would imagine that most people would learn to use this tool very quickly Strongly disagree	Somewhat disagreeI would imagine that most people would learn to use this tool very quickly Somewhat disagree	Neither agree nor disagreeI would imagine that most people would learn to use this tool very quickly Neither agree nor disagree	Somewhat agreeI would imagine that most people would learn to use this tool very quickly Somewhat agree	Strongly agreeI would imagine that most people would learn to use this tool very quickly Strongly agree
I found the tool very cumbersome to use	Strongly disagreeI found the tool very cumbersome to use Strongly disagree	Somewhat disagreeI found the tool very cumbersome to use Somewhat disagree	Neither agree nor disagreeI found the tool very cumbersome to use Neither agree nor disagree	Somewhat agreeI found the tool very cumbersome to use Somewhat agree	Strongly agreeI found the tool very cumbersome to use Strongly agree
I felt very confident using the tool	Strongly disagreeI felt very confident using the tool Strongly disagree	Somewhat disagreeI felt very confident using the tool Somewhat disagree	Neither agree nor disagreeI felt very confident using the tool Neither agree nor disagree	Somewhat agreeI felt very confident using the tool Somewhat agree	Strongly agreeI felt very confident using the tool Strongly agree
I needed to learn a lot of things before I could use this tool	Strongly disagreeI needed to learn a lot of things before I could use this tool Strongly disagree	Somewhat disagreeI needed to learn a lot of things before I could use this tool Somewhat disagree	Neither agree nor disagreeI needed to learn a lot of things before I could use this tool Neither agree nor disagree	Somewhat agreeI needed to learn a lot of things before I could use this tool Somewhat agree	Strongly agreeI needed to learn a lot of things before I could use this tool Strongly agree

Annotation tool functionalities. *

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
It was easy to use the tool for annotating the document.	Strongly DisagreeIt was easy to use the tool for annotating the document. Strongly Disagree	DisagreeIt was easy to use the tool for annotating the document. Disagree	NeutralIt was easy to use the tool for annotating the document. Neutral	AgreeIt was easy to use the tool for annotating the document. Agree	Strongly AgreeIt was easy to use the tool for annotating the document. Strongly Agree
It was easy to use the tool for comparing two documents.	Strongly DisagreeIt was easy to use the tool for comparing two documents. Strongly Disagree	DisagreeIt was easy to use the tool for comparing two documents. Disagree	NeutralIt was easy to use the tool for comparing two documents. Neutral	AgreeIt was easy to use the tool for comparing two documents. Agree	Strongly AgreeIt was easy to use the tool for comparing two documents. Strongly Agree
It was easy to use the tool for producing visualisations.	Strongly DisagreeIt was easy to use the tool for producing visualisations. Strongly Disagree	DisagreeIt was easy to use the tool for producing visualisations. Disagree	NeutralIt was easy to use the tool for producing visualisations. Neutral	AgreeIt was easy to use the tool for producing visualisations. Agree	Strongly AgreeIt was easy to use the tool for producing visualisations. Strongly Agree

The information provided by the tool was enough to perform the following tasks:

	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
Annotation	Strongly DisagreeAnnotation Strongly Disagree	DisagreeAnnotation Disagree	NeutralAnnotation Neutral	AgreeAnnotation Agree	Strongly AgreeAnnotation Strongly Agree
Document comparison	Strongly DisagreeDocument comparison Strongly Disagree	DisagreeDocument comparison Disagree	NeutralDocument comparison Neutral	AgreeDocument comparison Agree	Strongly AgreeDocument comparison Strongly Agree
Visualisations	Strongly DisagreeVisualisations Strongly Disagree	DisagreeVisualisations Disagree	NeutralVisualisations Neutral	AgreeVisualisations Agree	Strongly AgreeVisualisations Strongly Agree

How could the information provided be improved? *

List one or more aspects that positively affected your experience while using the VAST annotation tool: *

List one or more aspects that negatively affected your experience while using the tool: *

Do you have suggestions how to improve the overall usability of the tool? *

Do you have suggestions about possible functionalities that can be added to the tool? *

Cookie	Duration	Description
cookielawinfo-checkbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy	11 months	The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.