modal

K-Dense-AI/scientific-agent-skillsCloud PlatformsAudit failedSnapshot 6e9f4dd10b0d

Installs: 0

Summary

Modal is a serverless cloud platform for running Python on demand, including on-demand GPUs. Use when deploying or serving AI/ML models, running GPU-accelerated workloads (training, fine-tuning, inference), serving web endpoints, scheduling batch jobs, or scaling Python code to cloud containers with the Modal SDK.

SKILL.md

Modal

Overview

Modal is a cloud platform for running Python code serverlessly, with a focus on AI/ML workloads. Key capabilities:

GPU compute on demand (T4, L4, A10, L40S, A100, H100, H200, B200)
Serverless functions with autoscaling from zero to thousands of containers
Custom container images built entirely in Python code
Persistent storage via Volumes for model weights and datasets
Web endpoints for serving models and APIs
Scheduled jobs via cron or fixed intervals
Sub-second cold starts for low-latency inference

Everything in Modal is defined as code — no YAML, no Dockerfiles required (though both are supported).

When to Use This Skill

Use this skill when:

Deploy or serve AI/ML models in the cloud
Run GPU-accelerated computations (training, inference, fine-tuning)
Create serverless web APIs or endpoints
Scale batch processing jobs in parallel
Schedule recurring tasks (data pipelines, retraining, scraping)
Need persistent cloud storage for model weights or datasets
Want to run code in custom container environments
Build job queues or async task processing systems

Installation and Authentication

Install

uv pip install modal

The Modal Python SDK supports Python 3.10–3.14. This skill targets the stable modal>=1.0 API (current release: 1.4.x).

Authenticate

Prefer existing credentials before creating new ones. Only the two Modal-specific variables below are relevant — do not read, load, or expose any other environment variables or file contents:

.env

modal setup

export MODAL_TOKEN_ID=<your-token-id>
export MODAL_TOKEN_SECRET=<your-token-secret>

import modal

app = modal.App("my-app")

@app.function()
def square(x):
    return x ** 2

@app.local_entrypoint()
def main():
    # .remote() runs in the cloud
    print(square.remote(42))

image = (
    modal.Image.debian_slim(python_version="3.11")
    .uv_pip_install("torch==2.12.0", "transformers==5.9.0", "accelerate==1.13.0")
    .apt_install("git")
)

@app.function(image=image)
def inference(prompt):
    from transformers import pipeline
    pipe = pipeline("text-generation", model="meta-llama/Llama-3-8B")
    return pipe(prompt)

@app.function(gpu="H100")
def train_model():
    import torch
    device = torch.device("cuda")
    # GPU training code here

# Multiple GPUs
@app.function(gpu="H100:4")
def distributed_training():
    ...

# GPU fallback chain
@app.function(gpu=["H100", "A100-80GB", "A100-40GB"])
def flexible_inference():
    ...

vol = modal.Volume.from_name("model-weights", create_if_missing=True)

@app.function(volumes={"/data": vol})
def save_model():
    # Write to the mounted path
    with open("/data/model.pt", "wb") as f:
        torch.save(model.state_dict(), f)

@app.function(volumes={"/data": vol})
def load_model():
    model.load_state_dict(torch.load("/data/model.pt"))

@app.function(secrets=[modal.Secret.from_name("my-api-keys")])
def call_api():
    import os
    api_key = os.environ["API_KEY"]
    # Use the key

@app.function()
@modal.fastapi_endpoint()
def predict(text: str):
    return {"result": model.predict(text)}

@app.function(schedule=modal.Cron("0 9 * * *"))  # Daily at 9 AM UTC
def daily_pipeline():
    # ETL, retraining, scraping, etc.
    ...

@app.function(schedule=modal.Period(hours=6))
def periodic_check():
    ...

@app.function(
    max_containers=100,    # Upper limit
    min_containers=2,      # Keep warm for low latency
    buffer_containers=5,   # Reserve capacity
    scaledown_window=300,  # Idle seconds before shutdown
)
def process(data):
    ...

results = list(process.map([item1, item2, item3, ...]))

@app.function()
@modal.concurrent(max_inputs=10, target_inputs=8)
async def handle_request(req):
    ...

Model = modal.Cls.from_name("my-app", "Model")
fast = Model.with_options(gpu="H200", max_containers=20)
fast().generate.remote(prompt)

@app.function(
    cpu=4.0,              # Physical cores (not vCPUs)
    memory=16384,         # MiB
    ephemeral_disk=51200, # MiB (up to 3 TiB)
    timeout=3600,         # Seconds
)
def heavy_computation():
    ...

@app.cls(gpu="L40S", image=image)
class Predictor:
    @modal.enter()
    def load_model(self):
        self.model = load_heavy_model()  # Runs once on container start

    @modal.method()
    def predict(self, text: str):
        return self.model(text)

    @modal.exit()
    def cleanup(self):
        ...  # Runs on container shutdown

app = modal.App.lookup("sandbox-demo", create_if_missing=True)

# Isolated container; restrict egress for untrusted workloads
sb = modal.Sandbox.create(
    app=app,
    image=modal.Image.debian_slim(),
    outbound_cidr_allowlist=["10.0.0.0/8"],
)

# Stream files in/out via the filesystem API (beta)
sb.filesystem.write_text("print(2 ** 10)\n", "/tmp/job.py")
contents = sb.filesystem.read_text("/tmp/job.py")

sb.terminate()

import modal

app = modal.App("llm-service")

image = (
    modal.Image.debian_slim(python_version="3.11")
    .uv_pip_install("vllm")
)

@app.cls(gpu="H100", image=image, min_containers=1)
class LLMService:
    @modal.enter()
    def load(self):
        from vllm import LLM
        self.llm = LLM(model="meta-llama/Llama-3-70B")

    @modal.method()
    @modal.fastapi_endpoint(method="POST")
    def generate(self, prompt: str, max_tokens: int = 256):
        outputs = self.llm.generate([prompt], max_tokens=max_tokens)
        return {"text": outputs[0].outputs[0].text}

app = modal.App("batch-pipeline")
vol = modal.Volume.from_name("pipeline-data", create_if_missing=True)

@app.function(volumes={"/data": vol}, cpu=4.0, memory=8192)
def process_chunk(chunk_id: int):
    import pandas as pd
    df = pd.read_parquet(f"/data/input/chunk_{chunk_id}.parquet")
    result = heavy_transform(df)
    result.to_parquet(f"/data/output/chunk_{chunk_id}.parquet")
    return len(result)

@app.local_entrypoint()
def main():
    chunk_ids = list(range(100))
    results = list(process_chunk.map(chunk_ids))
    print(f"Processed {sum(results)} total rows")

app = modal.App("etl-pipeline")

@app.function(
    schedule=modal.Cron("0 */6 * * *"),  # Every 6 hours
    secrets=[modal.Secret.from_name("db-credentials")],
)
def etl_job():
    import os
    db_url = os.environ["DATABASE_URL"]
    # Extract, transform, load
    ...

Command	Description
`modal setup`	Authenticate with Modal
`modal run script.py`	Run a script's local entrypoint
`modal serve script.py`	Dev server with hot reload
`modal deploy script.py`	Deploy to production
`modal volume ls <name>`	List files in a volume
`modal volume put <name> <file>`	Upload file to volume
`modal volume get <name> <file>`	Download file from volume
`modal secret create <name> K=V`	Create a secret
`modal secret list`	List secrets
`modal app list`	List deployed apps
`modal app stop <name>`	Stop a deployed app

modal

Summary

SKILL.md

Modal

Overview

When to Use This Skill

Installation and Authentication

Install

Authenticate

Core Concepts

App and Functions

Container Images

GPU Compute

Volumes (Persistent Storage)

Secrets

Web Endpoints

Scheduled Jobs

Scaling and Concurrency

Resource Configuration

Classes with Lifecycle Hooks

Sandboxes

Common Workflow Patterns

GPU Model Inference Service

Batch Processing Pipeline

Scheduled Data Pipeline

CLI Reference

Security Notes

Reference Files

Related skills