Input Configurator FAQ
What is Input Configurator ?
ICON (Input Configurator) is a novel system for input-reconfigurable interactive applications. It is based on a reactive data-flow architecture that describes input methods using interconnected modules. A visual editor allows the developer to test and validate a large number of input methods for his application, depending on hardware, user and environment issues. ICON-aware applications can also be fully customized by power users.
For a more detailed explanation, see the short paper on ICON web page.
What about the readability of the visual language ? Can I create complex input configurations with ICON ?
It depends on what you call complex. Many behaviors of today's interactive applications are 'complex' in the sense that they are making use of intricate, highly modal and contextual interaction mechanisms. This is a consequence of using poor and generic input devices for controlling anything. Using ICON for modelling the full input behavior of today's WIMP applications would certainly lead to a very complex and poorly readable input configuration. In contrast, ICON is well-adapted to direct, parallel, input-rich interaction. It allows to build simple but powerful input techniques, and supports most modern post-WIMP paradigms, such as see-through tools.
Can I control an existing Java application with ICON ?
'Surface' control of existing Java applications is possible through Swing devices, without modifying the code. However, input configurability is far richer when you develop an application with ICON in mind. An existing application can be modified for partial ICON support, by implementing and externalizing a set of application devices. For a big application that has not been designed for concurrent interaction, full ICON support would be hard to achieve without modifying a great part of the code. We are currently working on a both deep and generic integration of ICON with the Jazz/Piccolo tookit.
Can I control an existing non-Java application with ICON ?
This is partly possible, because ICON is toolkit-independent. However, you are loosing the overlay feedback features of ICON, and you won't be able to disable default input behavior of the application. Moreover, control of existing non-Java application is made difficult due to the language barrier. As a proof of concept however, we developed a Word device in java that sends COM messages to the active MSWord document. This device sends cut/paste, and page change commands, and controls the current zoom factor of the document. This is a quite simple device, but which allows very interesting features, such as voice control or analog zoom control with a dedicated device.
Features of current ICON distribution
What is the status of current ICON distribution ?
The current ICON distribution is v0.2. Although it is fully functional, it has no decent documentation and some features still need to be implemented or debugged. This early distribution is not mature enough to be used as a development tool for commercial applications, but can however be used for prototyping interactive applications and testing input methods.
Under which license is ICON distributed ?
The current license is available at http://www.emn.fr/dragicevic/ICON/license.html.
This license may evolve to open source.
Is ICON fully documented ?
Not yet. The only documentation today consists in this FAQ, the readme.txt file that goes with the distribution and the javadoc files.
Is ICON 100% pure Java ?
ICON is pure Java, except for input device handling which uses native code. Access to input devices depend on the platform, and is currently better supported under MS Windows. However, ICON runs on any other platform, except that you may not see the same devices in the 'extended' folder.
What are the input devices currently supported by ICON ?
Under MS Windows, ICON supports one mouse, one keyboard, multiple tablet devices, multiple 'gaming' devices such as joysticks and 6DOF SpaceMice, IBM speech recognizers as well as MIDI Controllers. ICON also has a MIDI Synthesizer output device. See the Installation section for details about how to make your devices work with ICON.
Under X Windows systems, a support for XInput Extension devices has been written by Stéphane Conversy, though it has not been thoroughly tested.
Are you planning to support more devices ?
Yes. Our primary goal is to support all HID (USB) devices under most platforms. There are several projects of multiplatform USB support in java.
Does ICON supports USB's hot plug ?
Not yet, but we will add a 'refresh' option to the folder's pane. At now, you must restart ICON if you have plugged a new device.
Some implementation details
How does ICON handles mouse and keyboard devices ?
Mouse and keyboard devices have a very different implementation. The Keyboard device captures keyboard events from the Java event queue. It works on every platform, but you are not guaranteed to receive all keyboard events, especially if your java windows does not have keyboard focus.
The Mouse device uses the native DirectInput API to directly read the events from the mouse, independently from the focused window. It will send you delta values (dx, dy) from the mouse instead of cursor locations. This is an advantage if you don't want to depend on screen bounds. The bad news is that DirectInput's mouse still depends on the system cursor, and may send events coming from other pointing devices (e.g. a tablet device that is controlling the system cursor). You can prevent tablet devices from controlling the system cursor by dragging them into your configuration.
The native mouse device is only available under Windows platforms. We are planning to add a Java mouse device (similar to the Keyboard device) that works under any platform.
Is standard keyboard/mouse event handling deactivated while ICON is running ?
Yes, if you are using those devices in your configuration. The Keyboard device consumes all keyboard events from the java event queue, except the Alt+C keystroke which stops the configuration. The Mouse device disables both Java's handling of mouse events and the system cursor. The system cursor is disabled using a java hack, by hiding it and keeping it inside the java window. When the configuration is stopped, the system cursor comes back. To allow standard event handling in a configuration, uncheck the
disableSystem option on these devices.
How does ICON handles multiple cursors ?
The Cursor device uses a custom AlternateCursor class to animate cursors on top of java windows. One or more AlternateCursors can be installed on top of any Java window without the need of modifying the application code. They can be substituted to the system cursor without major difference for the user, except that they can't be displayed outside the window. Also note that Cursor devices are only feedback devices, they do not produce events by themselves. You must connect them to Swing devices to be able to control Swing widgets. Swing will react pretty well to concurrent mouse events, but some interactions will cause unexpected behavior. We are currently working on a clean integration with another graphical toolkit, Jazz/Piccolo.
How do I install ICON ?
Just unzip the distribution file in the folder of your choice. MS-DOS and Windows scripts are provided for running precompiled examples.
Are there some examples I can run to test my installation ?
- ICONDraw: Launch ICONDraw, then select 'Configure' in the Options menu. Load a configuration from the editor, e.g. 'ICONdraw_Default.ic', then start it. Your mouse cursor will be at the top left corner of the screen, so mouve the mouse toward the middle of the screen to see your cursor appear in ICONDraw window.
- HiNote: This example starts a Jazz demo application called HiNote. Standard jazz input handling is used by default. Load a SVG document from HiNote (look inside images/) then go to the editor window and load a HiNote configuration to enable ICON.
- SwingSet and SwingSet2: Those are standard Swing demos you can use to test Swing-compatible input configurations.
- InputEditor: You can launch the editor as a stand-alone application (i.e. without any application to control). You will be able to test your devices or control global output devices such as the midi synthesizer.
How can I use tablet devices (digitizers) with ICON ? [Windows]
ICON supports tablet devices through the Wintab API. Most tablet vendors, including Wacom, provide drivers that implement Wintab. If you have a Wacom tablet, follow those steps:
1. Install Wacom's tablet driver (this is the same driver for all tablets). Wacom's universal driver v4.60 is provided in the
redist directory. If you don't have any driver installed, it is strongly recommended you use this one, because we experimented problems with the latest driver versions (but only using ICON).
2. Register your tablet(s) with the Wacom driver. Serial tablets must be added manually by specifying their COM port. USB tablets are automatically added when plugged for the first time, though there can be detection problems. To avoid USB tablet detection problems after the driver has registred them, always plug them in the same port.
3. Launch ICON. You should see the tablet(s) listed in the
How can I use Joystick/Magellan/SpaceMouse devices with ICON ? [Windows]
ICON supports all directInput 'gaming' controllers that are listed in your 'game options' control panel.
1. Plug your device and make sure it is listed in the Windows's game options panel.
2. Important: Calibrate your device using this panel.
3. Launch ICON. You should see your device(s) in the
How can I use speech recognition with ICON ? [Windows]
ICON supports IBM implementation of JavaSpeech. You need to have a IBM ViaVoice speech recognizer installed.
1. Make sure you have ViaVoice installed, and properly trained.
2. Install IBM JavaSpeech implementation (ibmjs). You can download it from IBM website, or use the file provided in
ViaVoice/bin directories to your binary path.
4. Launch ICON. You should be able to use the
std.input.SpeechCmd device. Ctrl-click this device to set the speech vocabulary.
How can I use MIDI controllers with ICON ? [Windows]
Make sure that your MIDI input port(s) are listed in the Windows multimedia control panel. The same ports will appear in ICON in the
std.midi folder. Your controller(s) can't be explicitely detected but will be accessible through a unified interface.
Using the visual configurator
How do I use the visual configurator ?
There are currently no user documentation for the editor. However, the interactions are quite straightforward, except that selections are performed with the right mouse button. You may check the 'help' menu for the complete list of possible interactions.
Is the configurator input-configurable ?
Making the visual configurator fully ICON-aware is one of our projects.
Questions below are partly answered in the readme.txt file that you will find in last ICON distribution.
Developing with ICON
How are the sources organized ?
How to implement a processing device ?
How to implement an application device ?
How to implement an input device ?
How can I make my Java application ICON-aware ?
I'm currently writing a new Java application that I want to be ICON-aware. What are your designs tips ?
- Sometimes standard mouse pointer is not properly deactivated (Windows only).
- Handling of multiple documents (configurations) still has to be implemented.