| Date | Product / Name | Test Type & Methodology | User Type & Objectives | Key Findings / Design Decisions | Findings Report |
|---|
| 2020-09 | Work 10 Tree Advanced Features & Filter | - MBI customer review
- Remote Moderated
|
| - Refresh search/recent pages after the move so that the location data is updated
- Add an undo action after a move
- Add the ability to move the activities panel anywhere
- Auto-hide/collapse activities panel to keep it out of users’ way
- Add folder drag and drop within the tree
- Add a way to ‘X’ out individual filters
- Add a way to clear all applied filters
| Findings Report
Folder Tree & Filters MBI issues |
| 2020-02 | Work 10 Add to Folder | - MBI customer review
- Remote Moderated
|
|
|
|
| 2020-01 | Work 10 Advanced Search Findability Iteration #2 | - Usability test
- Remote unmoderated
- Clickable Hi-Fi Prototype
| - 8 Legal or Financial Services professionals
- End users compared two different designs for Advanced Search in the desktop and mobile views
- Users were tasked with finding out how to do a complex search
| Design Decisions: - An ‘Advanced’ text label will be added next to the arrow when a cursor is in the input field and simple search is activated.
- Interaction will continue to be built in so that both tapping the cursor onto the magnifying glass and into the search input field will activate search.
Future Design Considerations: - All test failures were because participants did not understand the difference between search criteria recommendations and advanced search input fields. Advanced search is the current priority, but future consideration may be taken to making simple search criteria clearer.
- On mobile, the dropdown icon is not universally recognized as an advanced search entry point. Due to space limitations and prior advanced search icon testing, a label or new icon will not be added for advanced search.
| Findings Report |
| 2019-11 | Work 10 Tree Phase 1 | - Customer Review
- Remote Moderated
- Live site
| - 5 iManage Work customers with varying job roles (Director of IT, Director of Information Governance, IT Project Manager, Records Manager, Litigation Support Manager)
- The organizations they represented were Stokes Lawrence, Latham & Watkins, Harper Macleod, Amazon, and Maschoff Brennan
- They were customers who had previously been interviewed and had requested the tree feature for Work 10
- 1 internal participant from the Cloud team
- 1 iManage Partner (Solutions Director from OIA)
| - Consider adding ‘Expand/Collapse’ menu command
- Change the name of Outlook Work Panel’s ‘Browse’ tab
- Consider adding an adjustable max-height gripper for each expandable section of the tree. This would allow users to keep some navigation items visible (e.g. My Favorites) instead of pushing everything below the fold.
| Findings Report |
| 2019-10 | Work 10 10.2 Benchmark | - Usability test
- Remote Unmoderated
- Live site
| - 6 internal users, 24 external users (Legal or Financial Services professionals)
- 20 tested on desktop, 10 on mobile devices
Features tested: Filter, Favorite, Simple Search, Copy, Move, Rename, Preview, Download, Upload New Version, Send Link, and general navigation
| What worked: What could be improved:
| Findings Report |
| 2019-08 | Work 10 Grid | - Customer Review
- Remote Moderated
- Clickable Hi-Fi Prototype
| - 4 iManage Work customers with varying job roles (Legal Operations Analyst, Global Records Program Manager, Legal Finance & Compliance Admin, and Paralegal)
- The organizations they represented were American Electric Power, Amazon, and Bankrate/Redventures
- They were all customers who had previously been interviewed and had requested the Grid feature for Work 10
| - Make “Tiny” view with alternating row fill the default. Enlarge the text of "Comfortable" view
- Remove or improve discoverability of how users can return to the default sort order
- Add right-click functionality to column headers
| Findings Report Presentation
|
| 2019-02 | Work 10 Advanced Search Findability | - Usability test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 12 Legal or Financial Services professionals
- End users compared three different designs for Advanced Search in the desktop and mobile views
- Users were tasked with finding out how to do a complex search
| - The Switch design with a ‘sticky’ selection and ‘Advanced Search’ label will be implemented.
| Findings Report |
| 2019-01 | Classic vs Work 10 | - Quantitative Performance Comparison test
- Remote Unmoderated
- Live site
| - 7 Classic and Work 10 users (internal)
- Users performed the same tasks on Classic DeskSite/FileSite & Work 10 Web/Outlook integration in order to compare click-count and time-on-task measurements
- Users performed 31 tasks (core tasks for document and email management) on each client. Users were given instructions on the most efficient path to complete those tasks.
| Click Count : - Work 10 required less clicks on 52% the tasks (i.e. 16 tasks) - these were tasks involving search, compare, filing email attachments, and opening an MS Office file
- Both Classic & Work 10 required an equal number of clicks on 32% of the of the tasks (i.e. 10 tasks)
- Classic required less clicks on 16% the tasks (i.e. 5 tasks) - these were tasks involving copy, move, and renaming an email when filing
Time on Task : - Work 10 was faster for tasks involving search, comparing documents, and filing emails
- Work 10 was slower for tasks involving f iling & renaming emails, previewing documents, opening documents, copying documents, and moving documents
| Findings Report |
| 2018-11 | Work 10 Outlook Integration Work Panel Advanced Search | - Usability test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 9 Legal or Financial Services professionals
- End users compared three different designs for Advanced Search in the Outlook Work panel - 'Dropdown' design, 'Settings Icon' design, and 'Edit Icon' design.
- Users were tasked with adding and changing the main criteria and its parameters to execute advanced search queries.
| - The 'Dropdown' design will be implemented (i.e. same design as for Work Web's Advanced Search)
- Consider moving the Advanced search icon up to a first-level affordance (rather than a second-level affordance tucked under the search field)
- Replace the minus icon (for removing Advanced Search criteria) with an 'X' icon
| Findings Report |
| 2018-09 | Work 10 Tree, Filter & Navigation (Design Comparison) | - Usability test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - User type:
- 8 Legal or Financial Services Professionals (Part 1)
- 30 Legal or Financial Services Professionals (Quantitative 1)
- 13 Internal iManage Employees (Qualitative 2)
- End users compared two different designs for tree navigation - Vertical design (navigation between main menu and folder tree in left panel w/ filters on the right) vs. Horizontal design (toggle between filters tab and folder tree tab in left panel w/ persistent horizontal main menu)
| Key Findings: - Filters was easier to find on the right
- The persistent main menu in the horizontal navigation was preferred
- Navigation between filters and folder tree tabs in the horizontal design were difficult to discover
- There was no significant difference in performance between both designs when it came to perceived difficulty, time on task, task success and design preference.
Design Decisions: - A sticky main folder menu was added to the vertical design because users struggled to find a back arrow.
- The filter and folder tree were highlighted and made more prominent in the horizontal design.
- Recent Emails & Emails Filed By Me were pulled out of the filter panel in the horizontal design, since they were actually sub-navigation
| Findings Report
|
| 2017-12 | Work 10 Table Multiselect Action Bar (Design Comparison) | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 8 Legal or Financial Services professionals
- End users compared two different styles of multiselect action bars - dark bar vs. white bar
| The white multiselect action bar will be implemented |
|
| 2017-11 | Work 10 Timeline (Design Comparison) | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 7 Legal or Financial Services professionals
- End users compared two different Timeline designs - fully visible filters displayedvertically with colored action icons vs.dropdown filters displayed horizontally withblack & white action icons
| The fully visible filters displayed vertically with colored icons will be implemented. | Findings Report |
| 2017-09 | Work 10 Navigation (Design Comparison) | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 10 Legal or Financial Services professionals
- End users compared two different types of navigation - via the tabs in a static horizontalheader vs. via an expandable/collapsible vertical column
| The navigation via an expandable/collapsible vertical column will be implemented with the addition of a hover-over slide out/tooltip over the icons when columns are in their collapsed state. | Findings Report |
| 2017-09 | Work 10 Icon Compliance (Design Comparison) | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 8 Legal or Financial Services professionals
- End users compared two different icon designs for the Preview mode of the Properties panel (i.e. black background) - colored icons with a white outline vs. white icons with a dark outline
| The colored icons with a white outline will be implemented. | Findings Report |
| 2017-09 | Work 10 Shared Matter List | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 6 Legal or Financial Services professionals
- End users tested sharing their 'My Matters' with a colleague and changing their view from their own ‘My Matters’ list to their colleague’s ‘My Matters’ list
| The dropdown chevron used to toggle between users' ‘My Matters’ list will be made slightly more prominent. | Findings Report |
| 2017-09 | Work 10 Sorting | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 6 Legal or Financial Services professionals
- End users tested sorting a list of document search results to determine if they could discover the ‘Sort by’ dropdown and use it to sort the search results alphabetically by title, by date, and by relevance.
| - ‘Relevance’ will be kept as is for now (this option is for documents & emails only)
- ‘Name’ will be changed to:
- ‘Title (alphabetical)’ for documents
- ‘Subject (alphabetical)’ for emails
- ‘Alphabetical’ for clients/matters
- ‘Edit Date’ will be changed to ‘Date Edited’ (this option is for documents only)
| Findings Report |
| 2017-09 | Work 10 Access Level Terms (Design Comparison) | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 6 Legal or Financial Services professionals
- End users compared two different sets of terms for security access levels - ‘Read Only’ vs. ‘Guest’; ‘Read/Write’ vs. ‘Collaborator’; and ‘Full Access’ vs. ‘Co-Owner’
| The following terms for security access levels will be implemented: 'Read Only’, ‘Read/Write’, & ‘Full Access’ | Findings Report |
| 2017-08 | Work 10 Outlook Integration Email Filing | - Usability test
- Remote Moderated
- Live site
| | What worked: Users felt they could be as, or more, productive with Work 10 Panel alone vs. FileSite 9.3.x Users preferred the look-and-feel of the new UI with the Work 10 Panel First time users were able to intuitively complete tasks using the Work 10 Panel
What could be improved: User need visual cues to reassure them that an email they just drag & dropped is filed Users expect functional continuity with drag & drop (i.e. the ability to attach files to Outlook emails via drag & drop from Work 10 Panel) Users were confused when they couldn’t input their search directly in the ‘EMM bar’ EMM Toolbar felt cumbersome and less intuitive compared to the Work 10 Panel Need an unobtrusive/lightweight way within Outlook to introduce users Work 10’s features
| Findings Report |
| 2017-08 | Work 10 Document Context Menu | - Card Sort
- Remote Unmoderated
| - 30 Legal or Financial Services professionals
- End users performed an open card sort in order to re-evaluate the grouping of the menu items in the document context menu. They were asked to sort the cards into groups of like terms, and to name the groups.
| Design Decision: IT Admins will be allowed to customize both the contents and organization of the WR document context menu via the iManage Control Center console. The grouping below will be implemented as the default state of the WR document context menu. Key Findings: - ‘Copy’, ‘Move’, and ‘Delete’ were grouped together
- ‘Add to Favorites’ was placed in its own group
- ‘Publish to Share’ was grouped with ‘Copy Link’
- ‘Timeline’ was placed in its own group
- ‘Preview’, ‘Edit’, and ‘Open Read Only’ were placed at the top of the menu since they are the three main actions of DMS users
- The ‘More Actions’ group was created for the least-used actions of DMS users
- ‘Download’ and ‘Upload New Version’ were grouped together because these actions are related for DMS users
| Findings Report |
| 2017-08 | Work 10 Empty Folder Icon (Design Comparison) | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 9 Legal or Financial Services professionals
- End users compared two different designs for an empty folder icon - white closed folder vs. yellow & white open empty folder
| The white closed folder icon will be implemented. | Findings Report |
| 2017-08 | Work 10 Container Navigation (Design Comparison) | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 9 Legal or Financial Services professionals
- End users compared two different designs for navigating between containers on the same and different levels - dropdowns between breadcrumbs vs. tree view in the left column
| - Phase 1: The dropdowns between the breadcrumbs will be implemented due to development and time limitations
- Phase 2: The tree solution will be implemented as soon as possible. In addition to this user test, we have heard from multiple sources that the tree feature will make a Classic to Web migration feasible for some users; the inability to browse fully across levels made using only web a “non-starter”.
| Findings Report XSprint Demo |
| 2017-07 | Work 10 Advanced Search | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 8 Legal or Financial Services professionals
- End users tested the findability and ease-of-use of the advanced search flow on the WR desktop UI
| - No changes initially: users able to understand/use advanced search interface
- First-level criteria options were intuitive and well received by users. Consider how to better order and/or teach the added functionality (i.e. via Add Additional Criteria row).
- Consider better options for second-level criteria menu, either by using sub-options or re-ordering criteria in order of use
| Findings Report |
| 2017-06 | Work 10 Container Properties (Design Comparison) | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 8 Legal or Financial Services professionals
- End users compared two different designs for viewing the properties of an open container - in right panel vs. in popup window
| - Right-clicking the container’s name will display its context menu (where one of the options is Properties)
- The container’s properties will be displayed in a pop-up window
| Findings Report |
| 2017-05 | Work 10 Favorites Categories (Design Comparison) | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 12 Legal or Financial Services professionals
- End users compared three different designs for how to favorite a document into a particular category
| The design where Favorites categories and sub-categories expand vertically will be implemented |
|
| 2017-05 | Work 10 Outlook Integration New Folder (Design Comparison) | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 14 Legal or Financial Services professionals & Windows users
- End users compared three different designs for how to create a new folder
| The design where New Folder icon is located above the middle panel and once user clicks it, the new folder appears in the right hand panel to be named and created will be implemented | Findings Report |
| 2017-04 | Work 10 Outlook Integration | - Usability test
- Remote Moderated
- Live site
| | What worked: Users found the ability to edit data from the Properties panel very useful All users were able to discover the Save as New Version icon Users found the new search to be a marked improvement over current search All users discovered the Send iManage Link icon. All users noticed the Outlook icon flashing yellow in the toolbar after they clicking this icon All users were able to send and file an email to a particular folder All users were able to discover the Insert iManage Attachment icon, and navigate to Recent Folders to select a file
What could be improved: - Users expect iManage menu options on individual email attachments
- Users expect double-click behavior when selecting file location from EMM field
- Display important information prominently within Properties panel. Place important information at the very top or bottom of the Properties panel
- Keep icon labeling consistent for Save buttons
- Add Comments information in the pop-up menu for documents in backstage of MS Word
| Findings Report |
| 2017-03 | Work 10 Trash & Collections Icons (Design Comparison) | - Bi-Weekly test
- Remote Unmoderated
- Clickable Hi-Fi Prototype
| - 6 Legal or Financial Services professionals
- End users
- compared three different designs for placement of the trash icon - under the Main Menu, under the More menu, or under the Action menu
- compared three different designs for the collections icon
| - Trash icon will be placed under "More" tab (which will be located next to "Clients" tab)
- File Drawer icon will represent the temporary container to hold documents (i.e. Collection)
|
|
| 2017-03 | Work 10 | - Benchmark test
- Remote Unmoderated
- Live site
| | Key Findings: - Users struggled with figuring out how to use move a document. Some users expected to be given options (a la Windows) of which folder they could move to (a la Windows), and some users didn't understand why the Move button was disabled.
- Users experienced difficulty navigating to another folder. Many noticed, but did not use the breadcrumbs to navigate.
- Users liked the Timeline feature, but some users were confused by the many options within Timeline
- Saving the last search term the in search field was very beneficial, and allowed users navigate with less frustration
| Findings Report |
| 2017-02 | Work 10 New Features | - Usability test
- Remote Moderated
- Live site
| - 15 Legal or Financial Services professionals (5 were tested on Computer, 5 were tested on Tablet, 5 were tested on Smartphone)
- End users were given the following tasks to perform on Computers, Tablets or Smartphones on the WR 10 UI:
- Filter documents
- Check versions
- Open email attachments
- Email older version of document
- Rename matter, document
- Copy & move document
- Find security details, jurisdiction of document
- Edit document's class
| Key Findings: Most users were able to figure out the new copy functionality and transfer this learning to the move functionality. Most users were able to find information in the new Properties and Versions tabs, and discovered they could do inline editing in the Properties tab. Many users were not able to discover the Save button when editing a matter name. They were confused that matter name editing required this extra step not required elsewhere. This made them insecure about when items actually got saved or not. Smartphone and tablet users were confused when clicking the paperclip icon opened the Properties tab instead of the Attachments tab.
| Findings Report |
| 2016-11 | Work 9.4 | - Benchmark test
- Remote Unmoderated
- Live site
| - 15 Legal or Financial Services professionals (5 were tested on a Computer, 5 were tested on a Tablet, 5 were tested on a Smartphone)
- End users were given the following tasks to perform on Computers, Tablets, or Smartphones using White Rabbit version 9.4:
- Filter documents, emails, and matters
- Search documents and emails
- Check versions
- Open email attachments
- Navigate to matters
- Email document as link and attachment
- Search Personalized vs All
| Key Findings: - Most users had issues searching within silos (e.g. having to first specify document, email, or matters tabs, and having to select specific search options). When asked what frustrated them most and what they would improve, most users mentioned search.
- When they got the No Matches! message, users were only offered the solution of searching All, which never resolved their search issue (which was that they were looking within the wrong silo)
- Many users were able to successfully filter documents, emails, and matters, but smartphone users had some issues interacting with the filter dialog
| Findings Report |
Comments
0 comments
Please sign in to leave a comment.