[QTI] 'ListInteraction' as an extension to QTI

[Michael Pfähler]

Jim,

thanks for familiarizing yourself with the proposal of a listInteraction.
Your questions clearly point to interesting issues that should be discussed
during the specification process. Furthermore, they helped me with
explicitly stating my implicit assumptions. At the moment, I'll try to
answer your questions from my point of view. This does not mean that there
are no other solutions to these questions. I attached my ideas below your
corresponding questions.

On Wed, Sep 24, 2008 at 11:06 PM, Jim Saiya <Jim.Saiya at aaae.org> wrote:

>  Michael,
>
>
>
> Great work on this interaction type.  I have some questions on the synonym
> feature and how it might be handled during parsing and evaluation.
>
>
>
> Would more than two choices be allowed to be related in a 'synonym set?'
>
For simplicity's sake, we were thinking of only one synonym per choice.
In order to find a good solution to this issue, I think we first need to
define whether the relation of synonyms is supposed to be uni- or
bidirectional.
(I'll use "A -> B" as an notation for "A having the same meaning as B".)
So, if it is unidirectional, then A -> B does not imply B -> A and it would
make sense to allow more than one choice as a synonym.
If it is thought to be bidirectional then A -> B and A -> C implies that
B->C and it could be modeled with only one synonym reference per choice.
To be honest - I don't know, whether domains with such coherences exist. In
order to be equipped for any challenge, it is probably better to allow more
than one choice.

>  Would a question writer be able to use any one of the members of a
> synonym set in the correctResponse element?  Say, opposed to only the 'root'
> choice that does not (or maybe does) have the synonym attribute.
>
Again, we need to define whether our synonym relation is bidirectional or
not. We thought of a bidirectional relationship.
If it is bidirectional, any member of a synonym set can be used within the
responseDeclaration. I'd suggest that there actually is one preferred choice
that does not have the synonym reference. If this choice is declared as a
right answer, the evaluating system needs to check whether the candidates
reply IS this choice or, if it is not, whether the candidate's choice is
synonymous to the declared answer. If a choice with a synonym reference
itself is specified within the responseDeclaration, the evaluating system
must follow this dependency and use the preferred choice instead.
If the relation between two choices is unidirectional, then there can be a
difference depending on between which choice is choosen within the
responseDeclaration. An Example: A -> B.
If A is correct, then only "A" is the correct answer while B is wrong. If B
is correct, then A and B are both correct answers.

>  Would all synonyms have to point to a 'root' choice?  In other words,
> what would happen if two or more synonyms point to one another creating a
> circular reference?
>
Yes, in our case the maximum length of a path is 1. You're definitly right
in your statement that a more complex definition could lead to circular
references that would have to be identified by the evaluating system
(potentially not trivial, esp. with more than one synonym identifier
allowed)

> What if the choice referenced in a hiddenChoice element was a member of a
> synonym set?  Would that exclude all of the members in the set, or just that
> one?  I don't see why you would want to block only one, but it might have a
> purpose.
>
Until now, we actually thought of this to hide only one choice and not the
other members of the set. But it then needs clarification on whether the
suppression of a preferred choice destroys the other choices relation or
not. Therefore, it makes sense that the removal of one element of a synonym
group supresses all other synonymous choices as well. If they are required
though, they can be added manually with an extraChoice.

>  I thought I'd just say these out loud while I'm thinking about it so they
> could be covered in the spec.  ;-)
>
Thank you for this. I think there is definitly room left for discussion. I'd
be very much interested in what other people think about this relationship
between synonyms (uni- or bidirectional). I'd be glad to exchange opinions
with other developers and domain experts on this issue.

If you find any inconsistencies or problems with the our depicted solutions,
please feel free to comment on it. It's definitly worth clarifying.

Regards,
Michael

>  *Jim Saiya*
>
> Senior IET System Developer
>
>
>
> *American Association of Airport Executives*
>
> 601 Madison Street
>
> Suite 400
>
> Alexandria, VA 22314
>
> 703 824-0500, Ext. 199
>
> FAX: 703 820-1395
>
> jim.saiya at aaae.org
>
> www.aaae.org
>
>
>
>
>
> -----Original Message-----
> *From:* ims-qti-bounces at lists.ucles.org.uk [mailto:
> ims-qti-bounces at lists.ucles.org.uk] *On Behalf Of *Michael Pfähler
> *Sent:* Monday, 22 September, 2008 4:43 AM
> *To:* ims-qti at lists.ucles.org.uk
> *Subject:* Re: [QTI] 'ListInteraction' as an extension to QTI
>
>
>
> Steve,
>
> thank you very much for your answer. I've now officially suggested the
> listInteraction to IMSGlobal.
> It surely will take time until it eventually finds its way into the QTI
> specification.
>
> As there are probably many developers on this mailing list, that are
> potentially interested, please read on for my suggestion of a
> 'listInteraction'.
> ( A potential rendering<http://home.in.tum.de/%7Epfaehlem/simpleregexsearch.html>of a listInteraction (uses javascript))
>
> If the time for the assessment is limited, the candidate does not have
> enough time to read all potential choices. Therefore, the candidate has to
> be aware of the correct answer in order to be able to answer the question
> correctly. Nevertheless, such a question can be easily evaluated.
> This question type therefore combines the strengths of freetext and
> multiple choice questions.
>
> The potential QTI data format that we propose for a listInteraction is as
> follows:
> (For clarity, standard qti namespace declarations and attributes are not
> mentioned within the examples)
>
> Within a Content Package that contains a listInteraction, you'll find these
> resources at minimum:
> - an AssessmentItem that contains a listInteraction
> - a dedicated xml file containing a list of many choices. This list can be
> referenced by several assessmentItems
> (- imsmanifest.xml :))
>
> - the list xml file is straight-forward and could look like this:
>
> <choiceList identifier="l39" >
>     <choice identifier="l39a1">Abdomen</choice>
>     <choice identifier="l39a2" synonym="l39a3">Appendix</choice>
>     <choice identifier="l39a3">Blind gut</choice>
>       …
>     <choice identifier="l39a603">Virus hepatitis</choice>
>     <choice identifier="l39a604">Virus infection</choice>
> </choiceList>
>
> The attribute 'synonym" allows synonymous entries to be referenced.
>
> - The listInteraction is embedded as a blockElement within the
> assessmentItem:
>
> <assessmentItem>
>    <responseDeclaration cardinality="single" identifier="q16030"
> baseType="identifier">
>         <correctResponse>
>            <value>l39a16</value>
>        </correctResponse>
>    </responseDeclaration>
>    <itemBody>
>       <listInteraction responseIdentifier="q16030" maxChoices="1"
> listIdentifier="l39" listHref="list39.xml">
>          <prompt>Which laboratory value determines the best differential
> diagnosis?</prompt>
>          <extraChoice identifier="q16030a1">xyz factor</extraChoice>
>          <hiddenChoice identifier="l39a261" />
>       </listInteraction>
>   </itemBody>
> </assessmentItem>
>
> While the list can be referenced by several assessmentItems, the
> responseDeclaration is specific to the interaction, i.e. a list can be
> reused amongst several items that require different correct answers. Within
> the listInteraction new choices can be added on a "per interaction basis"
> ('extraChoice'). It is also possible to blank out choices from the list for
> a specific question as they may be confusing within the question's context
> ('hiddenChoice'). However, this addition/suppression of choices on an
> interaction basis is optional.
> The listInteraction references its corresponding list by an identifier and
> by a hyperlink. This may be redundant but I can imagine it to be useful in
> some cases.
>
> - The list should be mentioned as a resource within the content package
> description. Maybe 'imsqti_answerlist_xmlv2p2' could be used as value for
> the attribute 'type' (in case it find's its way into QTI 2.2 :)).
>
> ... and this is it! Of course, such a list could be modeled simply as a
> ChoiceInteraction. However, you can not semantically define synonyms and
> redundancy is high if you reuse the answer list across several
> assessmentItems.
>
> We are currently using a customInteraction to express this question type in
> QTI. If you are interested in finding this question type within the official
> QTI specification, please let me know. Please share your ideas and thoughts
> with all through this mailing list.
>
> Thanks,
>
> Michael
>
>  On Tue, Sep 16, 2008 at 9:03 AM, Steve Lay <swl10 at cam.ac.uk> wrote:
>
>  Michael,
>
> Sorry this reply is a bit slow.
>
> The QTI specification is developed and maintained by the IMS Global
> Learning Consortium (IMS GLC).  The correct way to submit a request for
> future inclusion, or to report a problem, is to use the "Specification
> Problem and Suggestion Reporting" option on the IMS website.  It is under
> the "SPECIFICATIONS" menu or you can go straight there:
>
> http://www.imsglobal.org/problemtracking/index.cfm
>
> You will need to set up an ID on the system to make the suggestion but
> there is no cost involved and it only takes a few moment so please don't be
> put off by the initial redirect!
>
> It is also a great idea to discuss possible extensions, such as new
> interaction types, with a list like this though.  Even if ideas are not
> incorporated into the specification quickly (these processes tend to be slow
> for good reasons) there is nothing stopping you publishing information about
> your extension and trying to gain support amongst other developers.  In
> fact, exactly what you are doing!
>
> I suggest you post some examples of the proposed mark up in action, and
> perhaps point people to screen shots of what the interaction looks like in
> practice.
>
> Steve
>
> - -
>
> Michael Pfähler wrote:
>
>  Hello everybody,
>
> I'm working as a developer for an item banking system for medical items
> that is being developed by three German universities ( www.ims-m.org <
> http://www.ims-m.org/> ). We are trying to use QTI 2.1 as an interface to
> other systems, e.g. to assessment systems.
>
>
>
> Our item banking system manages a set of different question types. One of
> them is called a 'long menu question'. The candidate chooses the correct
> answers to a long menu question from a large list of answers (>500), so that
> such a question is both easily evaluable and at the same time avoids easy
> recognition of the correct answer.
>
> Syntactically such a long menu question could simply be expressed as a
> choiceInteraction. However, there are semantic differences (definition of
> synonyms, sharing a list across items). We currently use a customInteraction
> to export long menu questions. However, we think that the QTI standard could
> benefit from the integration of a dedicated 'listInteraction'.
>
> We already detailed a data format for this 'listInteraction' that I would
> like to share with you.
>
> I would greatly appreciate if you could point me to the appropriate entity
> that is responsible for such suggestions and at the same time, I would like
> to hear your opinions on the potential inclusion of this new question type
> into the current or a future QTI specification.
>
> Thank you very much,
> Regards,
>
> Michael
>
>  ------------------------------------------------------------------------
>
> _______________________________________________
> IMS-QTI mailing list
> IMS-QTI at lists.ucles.org.uk
> http://lists.ucles.org.uk/lists/listinfo/ims-qti
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: /public/ims-qti/attachments/20080926/7ff64f68/attachment-0001.html