Tutorial: Workflow for a Coding Project – iCode Support

Much of the coding and analysis process can be done independently or with support from experienced team members. New users who plan to build and code a portfolio in iCode are encouraged to take OPA’s iCode training class: https://dpcpsi.nih.gov/eo/intranet/opa/training/classroom-training

Step 1. Collect data

First, collect your data from a search, crowdsourcing effort, Request for Information (RFI), or another source.

If the data comes from an iSearch query, document the query and any filters used in your methods documentation. Because users often create multiple iterations of projects or portfolios, it is common to refine searches over time. For example, an initial search may use a first set of keywords, followed by subsequent refinements. Each iteration should be documented, along with the date the data was collected. View instructions for citing iSearch as a data source.

If possible, before collecting responses to an RFI or other data collection effort, plan the questions so that the answers can be grouped into categories. Free-text responses can be uploaded into iCode and are especially useful when the information does not fit into predefined categories.

If you are creating an RFI, contact your NIH Institute or Center’s RFI coordinator for assistance.

Step 2. Review data (pre-upload)

Review your data and ensure it includes all values required for upload into iCode. This step is not necessary if adding content to a project from an iSearch URL, but it is important when preparing spreadsheets for a project.

RFI responses and other spreadsheet data may need to be reformatted for consistency prior to being uploaded to the coding tool. You can perform some formatting in the tool, but briefly reviewing responses beforehand can help prevent issues later on. You do not need responses for every question on an RFI or other upload; it is okay to have blank cells. PDFs and other document types can be uploaded into the tool as well, but consider whether it would be more efficient to collect all the data in one central location prior to uploading it.

Note: To prepare your records and data for upload to iCode, maintain a master document where you continuously add RFI responses. When you upload multiple spreadsheets to iCode, the most recent spreadsheet will replace data from previously uploaded spreadsheets. To avoid losing data, make sure the new spreadsheet includes all previously uploaded data before uploading it. This is outlined in the Add Content/Records tutorial (more details in Step 3 below).

You need a unique identifier for each item, response, or row of data you intend to upload into the coding tool. For RFI responses, this may mean creating a column with a unique ID for each row. For search results, you can simply use the system ID from iSearch as a unique identifier.

Step 3. Create a project and upload your data in iCode

To create a project, select Create Project and then select Manage Content. Select Add Content to upload data to the new project. Follow the Add Content/Records tutorial for more details. During this process, you will upload source records and data, choose which fields to include, configure field formats, and select which fields will be available as filters. You can also create sections to organize related fields, making the data easier to view and navigate within the tool. To add or edit sections at any point, go to Modify View.

Step 4. Add collaborators

Select Add Collaborators in Settings to give other individuals access to the project. You can also add collaborators later. Collaborators can be Managers, Coders, or Viewers. The creator of the project automatically has Manager and Coder privileges.

Step 5. Review data in iCode (post-upload)

To get an overview of what you uploaded, go to the Results Table or Filters Screen. For example, you can see how many records have a specific activity code or linked data. Review any blank fields or unexpected numbers to ensure the data was uploaded and formatted correctly in iCode.

If you need to update the fields, go to Settings, select Manage Content to view the uploads, and then click Next to make your changes. Afterward, you can go to Modify View to manage or add sections to organize your data in the project's Record View. You can also reorder fields in Manage Sections.

Step 6. Code and analyze in iCode

Use the search bar to query your dataset in iCode. By default, any field you upload is searchable. You can change which fields are searchable in Modify View. The search bar supports the same advanced search functionality available in iSearch. Learn more about search syntax.

You can create coding fields at any time by following the Coding Fields and Vocabularies tutorial.

After adding your source data, you can also upload additional data as needed (see below), such as coder assignments. For step-by-step instructions, see the tutorials on assigning coders and how to code.

Note: You can use Upload Fields at any time to add coding or other data to records already in your project. The Upload Fields feature, located below the search bar and above the Results Table, allows you to add fields or tags to existing records. For example, you can add a field to assign coders or tag records with other distinctions.

Your upload file does not need to include all data previously uploaded to the project. See the Upload Fields Tutorial for more information.

If you need to change the organization or format of your fields or modify your coding fields, go to Settings in the left panel, then select Manage Content. Make these adjustments early in the coding process to help ensure consistent coding.

Coding review

If multiple team members are coding or reviewing records for relevance, it is important to apply the coding criteria consistently. Before coding begins, develop a codebook or coding protocol that clearly defines the criteria and ensures all team members are aligned.

It can also be helpful for team members to independently code a small sample of the same records and compare their results. This can identify inconsistencies, clarify ambiguous cases, and highlight scenarios that may need to be added to the codebook before proceeding with the full dataset.

Use the Review tab on the right side of the Record View screen to review coding. If you are a project Manager, you can also review coding completed by other coders.

Before beginning your analysis, consider assessing inter-rater reliability (IRR) to determine the level of agreement among coders. For example, IRR can help you assess whether coders consistently agree on which records should be included in the portfolio. OPA can assist with IRR; contact NIH-OPA_Support@nih.gov for assistance.

You can also use the Filters view and Charts view to explore your coding results. These views allow you to filter records and coding fields, view summary statistics, and interact with charts automatically generated for supported field types. They can also help you compare how records have been coded across your project. See the linked articles for more information.

Step 7. Export

You can export data from your project at any time. You can also download individual charts from the Charts view by selecting the download button within each chart. Exported data can be used to create additional visualizations or conduct further analysis outside of iCode.

To export your data, select the Export button below the search bar. You can export any field that was uploaded to the project or created as a coding field, along with each coder's response to that field.

After selecting the fields you want to include, submit the export request. When the export is ready, a notification will appear with an option to download the file. See the Export tutorial for more information.