Tutorial: Workflow for a coding project – iCode Support

Much of the coding and analysis process can be done independently or with support from experienced team members. New users who plan to build and code a portfolio in iCode are encouraged to take OPA’s iCode training class: https://dpcpsi.nih.gov/eo/intranet/opa/training/classroom-training

Step 1. Collect data

First, you'll want to collect your data, whether it is from a search, crowdsourcing, Request for Information (RFI), or another source.

If it is from an (iSearch) query/search, the query and/or filters should be recorded properly in a methods document. Many users create multiple iterations of project/portfolios, so it is common to take a first pass at the first query using keywords; this should be recorded in your documentation and also reflected in the date the data was collected. Instructions for citing iSearch as a data source are here: https://support.isearch.opa-tools.od.nih.gov/hc/en-us/articles/20487763707803-How-should-I-cite-iSearch-3-0

If possible, prior to collecting RFIs or other responses, plan the questions so the answers can be group into categories when possible. Free-text responses can be uploaded into iCode and are especially useful for coding responses that do not fit into predefined buckets or categorical classifications.

If you are creating an RFI, every NIH IC should have a coordinator to assist with this. Please reach out to this individual for assistance prior to creating your RFI.

Step 2. Review data (pre-upload)

Review your data and ensure you have all of the values necessary to upload into iCode. This is not necessary if adding content to a project from an iSearch url, but important for preparing spreadsheets for a project.

RFI responses and other spreadsheet data may need to be reformatted for consistency prior to being uploaded into the coding tool. You can do some formatting in the tool, but reviewing responses briefly beforehand can help prevent issues later on. You do not need responses for every question on an RFI or other upload; it is okay to have blank cells. PDFs and other document types can be uploaded into the tool as well, but you may want to consider if it is more efficient to collect all of the data in one central location prior to uploading to iCode.

Note: To prepare to upload your records and their data to iCode, keep a master document that you continuously add to for RFI responses. When uploading multiple spreadsheets to iCode, the most recent spreadsheet will replace previous spreadsheets. To avoid losing data, make sure the new spreadsheet includes all previously uploaded data before uploading it. This is outlined in the Add Content/Records tutorial (more details in Step 3 below).

You need a unique identifier for each item/response/row of data you intend to upload into the coding tool, however. This may mean for an RFI that you create a column with a unique ID. For uploading searches, a unique ID can easily be the system ID from iSearch.

Step 3. Create a project and upload your data in iCode

Create your project and select Manage Content to Add Content/Records to your new project. Follow the Add Content/Records tutorial for more details. This will include uploading your source records/data into the project, choosing fields to be included, setting the field formats and what fields to show as filters, along with the ability to create sections to group your fields into for ease of viewing in the tool (Visit Modify View to add and edit sections to best organize your project data at any point).

Step 4. Add Collaborators

Add other individuals you wish to have access to the project using the Add Collaborators setting. You can also do this at a later point. Collaborators can be Managers, Coders, or Viewers. The creator of the project automatically has Manager and Coding privileges.

Step 5. Review in iCode (post-upload)

To get an overview of what you uploaded, go to the Results Table, or Filters Screen. For example, you can see how many records have a specific activity code or linked data. Review any blanks or numbers that don't seem to align with what you expected to be in your data to ensure the data was uploaded/formatted correctly in iCode.

If you need to make updates to the fields/types, go back to your project Settings, and select Manage Content to view the uploads and click Next to make updates to field types and more. Afterwards, you can go to Modify View to Manage/Add Sections to organize your data in the Record view of your project if you wish; fields can be reordered to your choosing in Manage Sections as well.

Step 6. Code and analyze in iCode

Use the search bar to query your dataset in iCode. Any field uploaded will be searchable (this can be modified in Modify View) and the search bar has all of the complex search functionality as in iSearch. Learn more about search syntax. To make coding fields at any point, follow the coding fields and vocabularies tutorial. You may also want to upload additional data (see below), such as coder assignments at a later point, after adding your source data. Learn more about assigning coders and how to code in the relevant tutorials.

Note: You can use Upload Data (coding or additional data) at any point for the records that exist in your project by using the Upload Data feature. This feature (found in the project under the search bar, above the Results Table) allows you to add other fields and tags to your records at any point. This is often used to create an additional field to assign coders, or tag records in the project with other distinctions. You do not need to ensure you have all past uploaded data in the spreadsheet when using this option. See the Upload Data Tutorial for more information.

If you find that changes need to be made to the organization of your fields, field formats, or coding fields, go to Settings on the left panel of your screen (then Manage Content) and make adjustments. Be sure to do this early on in any coding exercise so that the coding is consistent.

Coding review

If you are coding or reviewing the data to see if the records you found are relevant, and have multiple people coding, you will want to be consistent in how you code. Prior to coding, your team should develop a coding book/protocol so that all members are aligned on the coding criteria. It is helpful to also code a handful of records amongst yourselves to ensure there are no problematic cases that are not covered in the coding book.

You can use the Review tab on the right hand side of the Record View screen to review coding done by yourself and if you are a Manager of the project, all coders. Prior to the analysis, you also will want to check your coding agreement to see if everyone agrees records belong in the portfolio. This is called IRR, or Inter-Rater Reliability. OPA can assist with IRR: contact us for assistance at: NIH-OPA_Support@nih.gov

You can use the Filters view and Charts view to apply filters, see summary statistics and interact with charts automatically created based on specific field types in your project. You can also see how coders have coded projects using these views, and filter on coding fields. Learn more in the relevant articles above.

Step 7. Export

You can download any data from your project at any time. Charts can be downloaded using the Charts view and selecting the download button within each chart. Any additional visualizations, data analysis and more can be done in the exported spreadsheet.

To export your data, use the Export button, which can be found just below the search bar. You can download any field that was uploaded or created as a coding field, along with every coder's response to that field. After choosing the fields to include in the export, an alert will appear in the notifications and you can download the file. Learn more about export here.