DMDX Help.

2D Input Device Keyword

<2DInputDevice text1 [options] text2,N1,N2,N3,N4 [...]>
<2Did text1 [options] text2,N1,N2,N3,N4 [...]>



built in devices:

    Parameter to select an input device as 2D input, ie a touch screen. text1 must be the name of an existing installed DirectInput device that has been calibrated by the TimeDX input test (unless it's "mouse""windowstouch" or "tcpipreply", see below). Once a device is selected there is no way to deselect it without starting a new item file (which can be done with the chain keyword), you can however unmap all the devices' buttons with <UnMapButtons>.  Names with spaces in them must be in quotes.  As of DMDX there a number of options that can follow  text1, notably datalogging, clicklogging, reltiveclicklogging, normalizedata, noclick, checkoverlapping, clipcursor, cursoroff and cursoron detailed below.  The  text2 is the arbitrary name for a region on the screen that will generate a response of that name (with added + and - symbols when hit or released) where N1 is the left most coordinate of the region,  N2 the top most, N3 is one more than the right most and N4 one more than the bottom (ie, a standard windows RECT structure).  text2 must begin with a non-numeric character.  text2,N1,N2,N3,N4 will need to be repeated for each possible region of the screen that will be used as an input throughout the whole item file.   N1,N2,N3,N4 can be normalized real values as per the graphics keywords assuming a video mode has already been specified.  If regions overlap the first defined region that matches the 2D coordinates is used (although checkoverlapping changes this, see below).  Coordinates of this RECT structure can be updated as the item file runs with <SetButtonRect> should your buttons move.  Alternatively if you wish to automate the setting of those regions instead of having to guess how large a target might be <sbr> when used in conjunction with <StoreCoords> will get the job done as detailed in the word rearranging demo at the end of this page as well as at the end of the Stroop section in the built in introduction, the text of which is available here.  In the following example even though the bottom input (specified in absolute coordinates whereas the others are fractional merely for demonstrative purposes) is totally overlapped it still catches touches in the bottom third of the screen:

<vm 1024,768,16,60> <2did "magic touch usb" bottom,0,512,1024,768 left,0,0,.5,1 right,.5,0,1,1> <mpr +right> <mnr +left> <mr +bottom>

    One problem used to be devices having cute little symbols as part of the name making it very hard to get the name of a device into an item file.  This should be mitigated in version of DMDX, please see the note about this in the input device section.


Built in devices:

    There are a few special built in devices for 2D input and these built in devices don't connect to any DirectInput device and instead either get their data from windows messages or other sources.  

    The first of them is the mouse device that taps into the windows mouse messages and does not need calibration in TimeDX (not that you could calibrate the mouse anyway as it has relative axes and the calibration routines only handle devices with absolute axes).  Both left and right clicks generate the same button press.  Because the mouse gets it's data from windows messages and those messages can get aggregated timing isn't going to be real accurate but as far as I can tell the whole pointing paradigm is alien to RTs in any event.  The mouse also turns the cursor on seeing as it's kinda hard to use a mouse to point at things unless there's a cursor...  Turning on DMDX's <MinSleep> option to maximize processing of the mouse system messages may increase RT accuracy of this special device.  Also note that the mouse device doesn't work real well with multiple monitor setups unless you're using the Direct3D rendering path.  When I wrote the code I assumed no one would ever need that but there you go.  The solution when using the classic DirectDraw renderer is to set the video card to clone mode where the two displays are copies of each other.  It's also possible setting a clipcursor region (below) in desktop coordinates might enable the mouse to be used on a second display.

    The second built in device is the windowstouch device added in version of DMDX.  Like the mouse device this device doesn't need calibration in TimeDX instead it  is calibrated somewhere in the Windows Control Panel.  Like the mouse the windowstouch device gets it's data from windows messages, specifically WM_TOUCH messages, however because it's dealing with isolated touches of the screen I suspect the timing of this device is much better.

    The third built in device is the tcpipreply device added in version of DMDX for interfacing with eye trackers.  This device does not need calibration in TimeDX as it is presumably calibrated elsewhere in the eye tracker software (or what have you, anything that talks line based ASCII TCP/IP can talk to the tcpipreply device).  The 2Did tcpipreply device parses data  from a TCP/IP reply setup with the <InputDevice> parameter and is thus unique needing both an <id> parameter and a <2Did> one.  The tcpipreply device has to be told the names of the fields that contain the data it uses so it must be followed by three strings as in tcpipreply,xfieldname,yfieldname,validstring.  The data in the first two fields are the x coordinate and the y coordinate from the eye tracker and can be real values between zero and one in which case they're multiplied by the relevant dimension of the screen.  In addition the device needs a particular validation string, validstring, to be present before the other fields can be considered valid and similar to the strings the <send> keyword sends it can contain '' for double quotes and ., for a semicolon.  If your device device doesn't contain a particular validation string per se pick some string that will always be present, it can be a single character.  See the <expireif> keyword help for an example of the tcpipreply device in use, typical use of tcpipreply for a Gazepoint eye tracker might  be tcpipreply,BPOGX,BPOGY,BPOGV=''1'' (so the BPOGV field is zero when the data isn't valid).



    The datalogging, clicklogging and relativeclicklogging options initiate data logging with each clockon, the original datalogging being a time based log and the new clicklogging (added in DMDX being a log entry with every click and relativeclicklogging (also added in logging coordinates relative to the upper left corner of the button clicked (as opposed to the other two which are relative to the top left corner of the display).  Only one of the three data logging options can be active at once and datalogging must be followed by two parameters, clicklogging and relativeclicklogging by one.  Firstly all three are followed by the maximum number of data points that could be gathered (not all points may be gathered in a trial, data logging terminates when a response is registered) and datalogging must have a second parameter, it being the period between each data point gathered in milliseconds.  Data logged are the coordinates that device presented at that time and when using datalogging not all devices provide a continuous data stream, a lot of them only report movement information so if the device isn't being moved you won't see any data points.  When using one of the click logging options only clicks that occur in button regions are logged (regardless of what DMDX does with the generated signals).  See the Bitmap notes for demo of relativeclicklogging.  Data is dumped at the end of each item on a single line in the output file in triplets of time,x,y and if you're using the "mouse" input device the times can be little wild as Windows delivers mouse movement information at it's leisure. 
    The last data point in any logging in an item is also saved in the macros .dataloggingx. and .dataloggingy. (as of DMDX opening up a whole new range of things one can do with DMDX as repositioning display elements by subjects is now in play -- although having just had a play around with it myself I needed <cco> to catch clicks between items however <cco> neuters data logging so if someone ever actually needs this I suspect modifications to DMDX will be in order.  As it stands having DMDX rearrange display elements is still doable, you just can't have the elements follow the cursor around the screen, you'd have to click on one once to nominate it for moving, then move the cursor to the new location and then a click  to deposit the element there, see the demo below.

    The normalizedata option (added in DMDX normalizes all data logged (so range 0 to 1), very handy when your item file will run on a number of different displays (not that you couldn't post hoc scale the data by the display resolution as it's emitted in the data file but this is muuuch easier).  For relativeclicklogging that normalized range is the button's dimensions, otherwise it's the whole display's dimensions.  normalizedata also affects the macros .dataloggingx. and .dataloggingy. so instead of being pixel integers they become real values with six significant digits (so 0.123456 for instance).

    The noclick option checks all data points received against the button regions so no clicking of buttons is needed.  Not all devices provide a continuous data stream so not all 2D devices can be used with noclick.

    The checkoverlapping option forces the routines that check coordinates from some device being within a given button's region to check all regions regardless of whether a match is found or not.  Normally DMDX stops once a match is found regardless of whether that region is mapped as a response or not.  Turning checkoverlapping on allows you to have multiple overlapping regions and select which one is active with the usual <mpr>, <mnr> etc. keywords instead of having to have lots of non overlapping regions and combining signals out of them (which was a pain, believe me).

    The cursoroff and cursoron options override the default windows cursor display state for that device.  Eg, if you are using the mouse device you'll need cursoroff if you don't want a cursor and you'll need cursoron for all other devices (to date anyway) if you want to see the cursor.  Note options that affect the cursor only work with devices that have drivers that affect the cursor.  This means things like touch screens and mice, joysticks won't affect the cursor (see Joysticks below).  If you need to change the cursor state while the job is running (ie after <2Did> has been given) because you're doing a multimodal task or want to indicate that an input isn't appropriate then the cursoroff and cursoron options are exposed in <mip> as well.

    The clipcursor option confines the cursor to given region of the screen when using the mouse 2Dinputdevice (read: no joystick support, see Joysticks below).  clipcursor must be followed by four coordinates in the same way as a button definition (see above)..  If you need to change the region as the job is running then the clipcursor option is exposed in <mip> as well.   Aside from uses I haven't thought of this allows continuous ratings to be made of video and logged with the datalogging option when the region provided to clip the cursor to is one pixel high and the width of the screen as the following item file demonstrates (a similar file dvRatingsDialdemo.rtf can be found in the digital video demo):

<ep> <cr> <dfm 0.55>
s3 <vm 800,600,16,60> <nfb> <2did mouse datalogging,50,50 clipcursor,0, 525, 800, 526>
<id keyboard>
$ 0 @-3 "The instructions", @-2 "across", @-1 "multiple", "lines" <branch 99>

mB+@14"0       1        2               3            4       5           6            7             8",

@15"    not at all/                             somewhat/                             extremely/",

@16"      none                                 some                             a great deal"+;

~99; $

+100 * <d 300> <t 4000> <dv -1,.1,.75,.75> "happy/429-53_l.mpg" <dv% 0> / ~B;
+101 * <d 300> <t 4000> <dv -1,.1,.75,.75> "neutral/485-31_l.mpg" <dv% 0> / ~B;
+102 * <d 300> <t 4000> <dv -1,.1,.75,.75> "sad/331-79_l.mpg" <dv% 0> / ~B;

$ 0 "Done"; $

     A more complicated use of the clipcursor option entails having the subject move the mouse back to the center of the screen between each rating (utilizing noclick so just positioning the cursor is enough).  Although joysticks can now do better job of this, see Joysticks below.  Note that <id keyboard> is no longer used as the request is provided by moving the mouse back to  the center of the screen:

<ep> <cr> <dfm 0.55>
s3 g2 <vm 800,600,16,60> <nfb> <2did mouse datalogging,50,50 noclick clipcursor,0,525,800,526 center,395,520,405,530> <mr +center>
$ 0 @-3 "The instructions", @-2 "this time", @-1 "the cursor", "needs to be centered" , @16 "Continue" <branch 99>

mB+@14"0       1        2               3            4       5           6            7             8",

@15"    not at all/                             somewhat/                             extremely/",

@16"      none                                 some                             a great deal"+;

~99; $

+100 * <d 300> <t 4000> <dv -1,.1,.75,.75> "happy/429-53_l.mpg" <dv% 0> / ~B;
0 @16 "Continue";
+101 * <d 300> <t 4000> <dv -1,.1,.75,.75> "neutral/485-31_l.mpg" <dv% 0> / ~B;
0 @16 "Continue";
+102 * <d 300> <t 4000> <dv -1,.1,.75,.75> "sad/331-79_l.mpg" <dv% 0> / ~B;
0 @16 "Continue";

$ 0 "Done";

     Note also that just leaving the cursor in the center of the screen won't automatically continue to the next item as Windows only provides data when the mouse moves so DMDX doesn't see the non-moving mouse parked over the request button, a rather nice side effect.

     This is the word rearranging task from above that uses clicklogging along with the .dataloggingx. and .dataloggingy. macros.  After setting up the initial positions of the words and a counter to remember what was clicked on last we display them in item 10 and store the coordinates they were displayed at with <sc> so that in item 20 we can set the word1 and word2 button RECTs with <sbr> allowing us to pick up clicks on them or the background (we've carefully arranged our 2Did buttons so the words will be checked first and only if they're not clicked on will we generate a signal for the rest of the screen being clicked on (with the nothing button) allowing us to detect where the previously selected word should be dropped).  Also in item 20 we have a <mwb> multi-way branch that splits execution based on what was clicked, if it was one of the words we remember it in our counter, if not we do an indexed branch with <ib> in item 300 that will cunningly not do anything if a word hadn't been selected already in item 400 otherwise in items 401 or 402 we set the coordinate macros of whatever word was selected to the last data logging location in the .dataloggingx. and .dataloggingy. macros and loop around to item 10 again (which will update the button RECTs and so on):

<fd 30> <cr> <id #keyboard> <ntl> <d 0> <nfb>
<2Did mouse clicklogging,10
<xyjustification framebyframe> <inst hardmargin>

0 <inst .2,.3,.7> "In ", "this ", "version ", "of ", "the ", "word ", "rearranging ", "task ", "click ", "on ", "a ", "word ", "(once) ", "and ", "then ", "click ", "elsewhere ", "to ", "move ", "it ", "there.";

~1 m.word1x.+.25+ m.word1y.+.5+
    m.word1l.+1+ m.word1t.+2+ m.word1r.+3+ m.word1b.+4+
    m.word2x.+.75+ m.word2y.+.5+
    m.word2l.+1+ m.word2t.+2+ m.word2r.+3+ m.word2b.+4+
    <mpr +word1> <mpr +word2> <mpr +nothing>
    <set c1=0>;

10 "word1" <xyjustification center> <xy ~.word1x.,~.word1y.>
        <sc .word1l., .word1t., .word1r., .word1b.>,
    "word2" <xyjustification center> <xy ~.word2x.,~.word2y.>
        <sc .word2l., .word2t., .word2r., .word2b.>;
+20 <sbr word1, ~.word1l., ~.word1t., ~.word1r., ~.word1b.>
    <sbr word2, ~.word2l., ~.word2t., ~.word2r., ~.word2b.> ! *
    <mwb +word1,100 +word2,200 +nothing,300 binr,300>;
~100 <set c1=1> <bu -10>;
~200 <set c1=2> <bu -10>;
~300 <set c2=400+c1> <set c1=0> <ib c2>;
~400 <bu -10>;
~401 m.word1x.+~.dataloggingx.+ m.word1y.+~.dataloggingy.+ <bu -10>;
~402 m.word2x.+~.dataloggingx.+ m.word2y.+~.dataloggingy.+ <bu -10>;

0 "Done";


     Joystick drivers don't manipulate the windows cursor so users are driving blind when using a joystick in DMDX.  If visual feedback of how far a stick is being pushed is required you can load up an external driver that takes the joystick and manipulates the cursor's position with it.  When used in absolute mode the cursor will always return to the center of the screen making rating tasks automatically home on a neutral rating (assuming of course you have positive and negative ratings).  We found a nice one called Joystick Mouse Tool 1.0 that works well.  You no longer tell <2Did> to use the joystick but instead use the mouse device mentioned above with no restrictions.

DMDX Index.