Record Importing & Exporting

Introduction

Data importing and exporting are essential features for research workflow integration. Whether you need to bulk upload existing research data, share datasets with colleagues, analyze data in specialized tools, or create backups, these features provide flexible options for data transfer and management.

This guide covers both importing CSV data to populate your models and exporting existing records for use in external applications.

Importing Records

The import feature allows you to upload CSV files to quickly populate your models with existing data. This is especially useful when migrating from other systems or when you have large datasets to enter.

When to Use Import

• Migrating from other systems: Transfer data from spreadsheets or other databases
• Bulk data entry: Add many records at once instead of entering them manually
• Collaborative data collection: Import data collected by team members offline
• Regular data updates: Import periodic datasets from ongoing research

Import Process

The import process is designed to be intuitive and provides validation feedback to ensure data quality. It consists of three main steps:

Step 1: Upload Your CSV File

1. Navigate to the Records page for the model you want to import data into
2. Click the "Import Records" button
3. Choose your CSV file using the file selector
4. The system will validate the file format and show you a preview of the first few rows

Step 2: Map CSV Columns to Attributes

In this step, you tell the system which columns in your CSV correspond to which attributes in your model:

• Column Preview: See sample data from each column to help identify what it contains
• Attribute Mapping: Use dropdown menus to map each CSV column to a model attribute
• Required Fields: Attributes marked as required must be mapped to proceed
• Skip Columns: You can choose to skip columns that aren't needed

Step 3: Review and Import

Before finalizing the import, the system validates all your data and shows you a summary. If there are validation errors, we've made it incredibly easy to fix them:

• Validation Results: See how many records are valid and how many have errors
• Data Preview: Review sample valid records to ensure they look correct
• Error Details: See specific validation errors for invalid records with clear, actionable messages
• One-Click Error Export: Download a CSV file containing only the problematic rows - no need to dig through your entire dataset to find what needs fixing
• Import Decision: Choose to import only valid records now, or fix errors and try again later

Data Validation

The import system automatically validates your data against the attribute types and validation rules you've defined for your model. This ensures data consistency and helps catch errors early.

Validation Types

Understanding Validation Results

After mapping your columns, the system processes your entire CSV file and categorizes each row:

• Valid Records: Pass all validation rules and are ready to import
• Invalid Records: Have one or more validation errors that must be fixed

For invalid records, you'll see:

• The specific row number that has errors
• Which fields have validation errors
• Clear error messages explaining what's wrong
• The actual data values that caused the errors

Handling Import Errors

When your import contains validation errors, you have several options for resolving them efficiently without starting over.

Download Invalid Rows

The most efficient way to fix errors is to download a CSV file containing only the invalid rows:

1. On the validation results page, click "Download Invalid Rows"
2. Open the downloaded CSV file in your preferred spreadsheet application
3. Fix the data issues based on the error messages you saw in the interface
4. Save the corrected file and import it as a new batch

Import Valid Records Only

If you want to proceed with the valid data while fixing errors separately:

1. Review the validation results to understand which records are valid
2. Click "Import Valid Records" to add only the error-free data to your model
3. Download the invalid rows to fix them separately
4. Import the corrected data later as a separate batch

Common Error Solutions

Exporting Records

You can export records to CSV format for use in spreadsheet applications, data analysis tools, or as backups:

Export Process

1. Navigate to the Records list for the model you want to export
2. Click the "Export Data" button in the upper right corner
3. In the export dialog, select the attributes you want to include
4. Choose whether to include related records' data
5. Click "Export" to generate and download the CSV file

The export will include:

• A header row with attribute names
• One row per record
• Values formatted appropriately for their data type
• Related data (if selected) in additional columns

Export Options

Customize your exports to include exactly the data you need:

Attribute Selection

• Select All: Include all attributes in the export
• Custom Selection: Choose specific attributes to include
• Reorder Attributes: Drag and drop to change column order in the export

Related Data

For records that have relationships with other models:

• Include Related Records: For each relationship type, select whether to include data from related records
• Choose Related Attributes: Select which attributes from related records to include
• Naming Convention: Related attributes are named with the format "relationshipName.attributeName"

Filtered Exports

Export only the data you're interested in:

• Apply filters to the records list before exporting to include only matching records
• Sort the records before exporting to maintain that order in the output file
• Use search to find specific records, then export just those results

Troubleshooting

Common issues and how to resolve them for both importing and exporting:

CSV File Requirements

For successful imports, ensure your CSV files meet these requirements:

• Header row: First row must contain column names
• Encoding: Save file as UTF-8 to preserve special characters
• File size: Maximum 10MB per upload
• Date format: Use YYYY-MM-DD format for date fields
• Number format: Use decimal points (not commas) for numbers
• Boolean values: Use "true"/"false", "yes"/"no", or "1"/"0"

System Limitations