Summary DNAnexus cloud genomics platform. Build apps/applets, manage data (upload/download), dxpy Python SDK, run workflows, FASTQ/BAM/VCF, for genomics pipeline development and execution.
DNAnexus Integration
Overview
DNAnexus is a cloud platform for biomedical data analysis and genomics. Build and deploy apps/applets, manage data objects, run workflows, and use the dxpy Python SDK for genomics pipeline development and execution.
When to Use This Skill
This skill should be used when:
Creating, building, or modifying DNAnexus apps/applets
Uploading, downloading, searching, or organizing files and records
Running analyses, monitoring jobs, creating workflows
Writing scripts using dxpy to interact with the platform
Setting up dxapp.json, managing dependencies, using Docker
Processing FASTQ, BAM, VCF, or other bioinformatics files
Managing projects, permissions, or platform resources
Core Capabilities
The skill is organized into five main areas, each with detailed reference documentation:
1. App Development
Purpose : Create executable programs (apps/applets) that run on the DNAnexus platform.
Key Operations :
Generate app skeleton with dx-app-wizard
Write Python or Bash apps with proper entry points
Handle input/output data objects
Deploy with dx build or dx build --app
Test apps on the platform
Common Use Cases :
Bioinformatics pipelines (alignment, variant calling)
Data processing workflows
Quality control and filtering
Format conversion tools
Reference : See references/app-development.md for:
Complete app structure and patterns
Python entry point decorators
Input/output handling with dxpy
Development best practices
Common issues and solutions
2. Data Operations Purpose : Manage files, records, and other data objects on the platform.
Upload/download files with dxpy.upload_local_file() and dxpy.download_dxfile()
Create and manage records with metadata
Search for data objects by name, properties, or type
Clone data between projects
Manage project folders and permissions
Uploading sequencing data (FASTQ files)
Organizing analysis results
Searching for specific samples or experiments
Backing up data across projects
Managing reference genomes and annotations
Reference : See references/data-operations.md for:
Complete file and record operations
Data object lifecycle (open/closed states)
Search and discovery patterns
Project management
Batch operations
3. Job Execution Purpose : Run analyses, monitor execution, and orchestrate workflows.
Launch jobs with applet.run() or app.run()
Monitor job status and logs
Create subjobs for parallel processing
Build and run multi-step workflows
Chain jobs with output references
Running genomics analyses on sequencing data
Parallel processing of multiple samples
Multi-step analysis pipelines
Monitoring long-running computations
Debugging failed jobs
Reference : See references/job-execution.md for:
Complete job lifecycle and states
Workflow creation and orchestration
Parallel execution patterns
Job monitoring and debugging
Resource management
4. Python SDK (dxpy) Purpose : Programmatic access to DNAnexus platform through Python.
Work with data object handlers (DXFile, DXRecord, DXApplet, etc.)
Use high-level functions for common tasks
Make direct API calls for advanced operations
Create links and references between objects
Search and discover platform resources
Automation scripts for data management
Custom analysis pipelines
Batch processing workflows
Integration with external tools
Data migration and organization
Reference : See references/python-sdk.md for:
Complete dxpy class reference
High-level utility functions
API method documentation
Error handling patterns
Common code patterns
5. Configuration and Dependencies Purpose : Configure app metadata and manage dependencies.
Write dxapp.json with inputs, outputs, and run specs
Install system packages (execDepends)
Bundle custom tools and resources
Use assets for shared dependencies
Integrate Docker containers
Configure instance types and timeouts
Defining app input/output specifications
Installing bioinformatics tools (samtools, bwa, etc.)
Managing Python package dependencies
Using Docker images for complex environments
Selecting computational resources
Reference : See references/configuration.md for:
Complete dxapp.json specification
Dependency management strategies
Docker integration patterns
Regional and resource configuration
Example configurations
Quick Start Examples
Upload and Analyze Data import dxpy
# Upload input file
input_file = dxpy.upload_local_file("sample.fastq", project="project-xxxx")
# Run analysis
job = dxpy.DXApplet("applet-xxxx").run({
"reads": dxpy.dxlink(input_file.get_id())
})
# Wait for completion
job.wait_on_done()
# Download results
output_id = job.describe()["output"]["aligned_reads"]["$dnanexus_link"]
dxpy.download_dxfile(output_id, "aligned.bam")
Search and Download Files import dxpy
# Find BAM files from a specific experiment
files = dxpy.find_data_objects(
classname="file",
name="*.bam",
properties={"experiment": "exp001"},
project="project-xxxx"
)
# Download each file
for file_result in files:
file_obj = dxpy.DXFile(file_result["id"])
filename = file_obj.describe()["name"]
dxpy.download_dxfile(file_result["id"], filename)
Create Simple App # src/my-app.py
import dxpy
import subprocess
@dxpy.entry_point('main')
def main(input_file, quality_threshold=30):
# Download input
dxpy.download_dxfile(input_file["$dnanexus_link"], "input.fastq")
# Process
subprocess.check_call([
"quality_filter",
"--input", "input.fastq",
"--output", "filtered.fastq",
"--threshold", str(quality_threshold)
])
# Upload output
output_file = dxpy.upload_local_file("filtered.fastq")
return {
"filtered_reads": dxpy.dxlink(output_file)
}
dxpy.run()
Workflow Decision Tree When working with DNAnexus, follow this decision tree:
Need to create a new executable?
Yes → Use App Development (references/app-development.md)
No → Continue to step 2
Need to manage files or data?
Yes → Use Data Operations (references/data-operations.md)
No → Continue to step 3
Need to run an analysis or workflow?
Yes → Use Job Execution (references/job-execution.md)
No → Continue to step 4
Writing Python scripts for automation?
Yes → Use Python SDK (references/python-sdk.md)
No → Continue to step 5
Configuring app settings or dependencies?
Yes → Use Configuration (references/configuration.md)
Often you'll need multiple capabilities together (e.g., app development + configuration, or data operations + job execution).
Installation and Authentication
Install dxpy
Login to DNAnexus This authenticates your session and sets up access to projects and data.
Verify Installation
Common Patterns
Pattern 1: Batch Processing Process multiple files with the same analysis:
# Find all FASTQ files
files = dxpy.find_data_objects(
classname="file",
name="*.fastq",
project="project-xxxx"
)
# Launch parallel jobs
jobs = []
for file_result in files:
job = dxpy.DXApplet("applet-xxxx").run({
"input": dxpy.dxlink(file_result["id"])
})
jobs.append(job)
# Wait for all completions
for job in jobs:
job.wait_on_done()
Pattern 2: Multi-Step Pipeline Chain multiple analyses together:
# Step 1: Quality control
qc_job = qc_applet.run({"reads": input_file})
# Step 2: Alignment (uses QC output)
align_job = align_applet.run({
"reads": qc_job.get_output_ref("filtered_reads")
})
# Step 3: Variant calling (uses alignment output)
variant_job = variant_applet.run({
"bam": align_job.get_output_ref("aligned_bam")
})
Pattern 3: Data Organization Organize analysis results systematically:
# Create organized folder structure
dxpy.api.project_new_folder(
"project-xxxx",
{"folder": "/experiments/exp001/results", "parents": True}
)
# Upload with metadata
result_file = dxpy.upload_local_file(
"results.txt",
project="project-xxxx",
folder="/experiments/exp001/results",
properties={
"experiment": "exp001",
"sample": "sample1",
"analysis_date": "2025-10-20"
},
tags=["validated", "published"]
)
Best Practices
Error Handling : Always wrap API calls in try-except blocksResource Management : Choose appropriate instance types for workloadsData Organization : Use consistent folder structures and metadataCost Optimization : Archive old data, use appropriate storage classesDocumentation : Include clear descriptions in dxapp.jsonTesting : Test apps with various input types before production useVersion Control : Use semantic versioning for appsSecurity : Never hardcode credentials in source codeLogging : Include informative log messages for debuggingCleanup : Remove temporary files and failed jobs
Resources This skill includes detailed reference documentation:
references/
app-development.md - Complete guide to building and deploying apps/appletsdata-operations.md - File management, records, search, and project operationsjob-execution.md - Running jobs, workflows, monitoring, and parallel processingpython-sdk.md - Comprehensive dxpy library reference with all classes and functionsconfiguration.md - dxapp.json specification and dependency management Load these references when you need detailed information about specific operations or when working on complex tasks.
Getting Help 