Batch Processing - Reprompt

Overview

The Batch Processing API allows you to enrich hundreds or thousands of places in a single request. Instead of making individual API calls for each place, submit them all at once and retrieve results efficiently.

Key Benefits

High Throughput - Process thousands of places in parallel
Cost Efficient - Reduced API overhead compared to individual requests
Progress Tracking - Monitor batch status and completion
Automatic Retries - Failed items are retried automatically
Result Management - Download complete or partial results

Quick Start

Create a Batch

Submit multiple places for enrichment in a single request:

const response = await fetch(
  'https://reprompt--reprompt-fastapi-fastapi-app.us-west.modal.run/{org_slug}/place_enrichment/batches',
  {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer YOUR_API_KEY',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      batch_name: 'Q1 2024 Store Locations',
      places: [
        {
          place_id: 'store_001',
          inputs: {
            name: 'Downtown Store',
            latitude: 40.7128,
            longitude: -74.0060
          }
        },
        {
          place_id: 'store_002',
          inputs: {
            name: 'Uptown Branch',
            address: '789 Broadway, New York, NY'
          }
        },
        // ... up to 1000 places per batch
      ]
    })
  }
);

const batch = await response.json();
console.log(`Batch created: ${batch.batch_id}`);

Monitor Progress

Check the status of your batch:

const statusResponse = await fetch(
  `https://reprompt--reprompt-fastapi-fastapi-app.us-west.modal.run/{org_slug}/place_enrichment/batches/${batch.batch_id}`,
  {
    headers: {
      'Authorization': 'Bearer YOUR_API_KEY'
    }
  }
);

const status = await statusResponse.json();
console.log(`Progress: ${status.completed_count}/${status.total_count}`);
console.log(`Status: ${status.status}`);

Retrieve Results

Once complete, download the enriched data:

if (status.status === 'completed') {
  const results = status.results;
  
  results.forEach(place => {
    console.log(`${place.place_id}: ${place.enrichment_status}`);
    if (place.enriched_data) {
      console.log('  Enriched attributes:', place.enriched_data);
    }
  });
}

Batch Limits

Maximum places per batch: 1,000
Maximum concurrent batches: 10 per organization
Batch timeout: 24 hours
Result retention: 30 days

For larger datasets (>1,000 places), split them into multiple batches or contact support for enterprise options.

Batch Status Lifecycle

queued - Batch created and waiting to process
processing - Enrichment in progress
completed - All places processed successfully
partial - Some places failed, but batch is complete
failed - Batch processing failed entirely

Monitoring Progress

Polling for Updates

async function waitForBatchCompletion(batchId) {
  const pollInterval = 5000; // 5 seconds
  
  while (true) {
    const response = await fetch(
      `https://reprompt--reprompt-fastapi-fastapi-app.us-west.modal.run/{org_slug}/place_enrichment/batches/${batchId}`,
      {
        headers: { 'Authorization': 'Bearer YOUR_API_KEY' }
      }
    );
    
    const status = await response.json();
    console.log(`Progress: ${status.completed_count}/${status.total_count}`);
    
    if (['completed', 'partial', 'failed'].includes(status.status)) {
      return status;
    }
    
    await new Promise(resolve => setTimeout(resolve, pollInterval));
  }
}

Progress Indicators

The batch status response includes:

{
  "batch_id": "batch_abc123",
  "batch_name": "Q1 2024 Store Locations",
  "status": "processing",
  "total_count": 500,
  "completed_count": 347,
  "failed_count": 3,
  "pending_count": 150,
  "created_at": "2024-01-15T10:00:00Z",
  "updated_at": "2024-01-15T10:15:23Z",
  "estimated_completion": "2024-01-15T10:25:00Z"
}

Handling Results

Success Results

const successfulPlaces = status.results.filter(
  place => place.enrichment_status === 'completed'
);

successfulPlaces.forEach(place => {
  // Process enriched data
  const enrichedData = place.enriched_data;
  updateDatabase(place.place_id, enrichedData);
});

Failed Items

const failedPlaces = status.results.filter(
  place => place.enrichment_status === 'failed'
);

failedPlaces.forEach(place => {
  console.error(`Failed to enrich ${place.place_id}:`, place.error_message);
  // Optionally retry or log for manual review
});

Reprocessing Failed Items

Retry specific places from a batch:

const response = await fetch(
  `https://reprompt--reprompt-fastapi-fastapi-app.us-west.modal.run/{org_slug}/place_enrichment/batches/${batchId}/reprocess`,
  {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer YOUR_API_KEY',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      place_ids: ['store_001', 'store_015', 'store_042']
    })
  }
);

List All Batches

Retrieve all batches for your organization:

const response = await fetch(
  'https://reprompt--reprompt-fastapi-fastapi-app.us-west.modal.run/{org_slug}/place_enrichment/batches',
  {
    headers: {
      'Authorization': 'Bearer YOUR_API_KEY'
    }
  }
);

const batches = await response.json();
batches.items.forEach(batch => {
  console.log(`${batch.batch_name}: ${batch.status} (${batch.completed_count}/${batch.total_count})`);
});

Best Practices

Optimize Batch Size: While you can submit up to 1,000 places per batch, smaller batches (100-300) complete faster and are easier to manage.

Use Descriptive Names: Give your batches meaningful names to track different datasets or processing runs.

Handle Failures Gracefully: Always check for failed items and implement retry logic for critical data.

Archive Results: Download and store batch results locally - they’re only retained for 30 days.

Common Patterns

CSV Upload Processing

import { parse } from 'csv-parse';
import fs from 'fs';

async function processCsvFile(filepath) {
  const places = [];
  
  // Parse CSV
  const parser = fs.createReadStream(filepath).pipe(parse({
    columns: true,
    skip_empty_lines: true
  }));
  
  for await (const row of parser) {
    places.push({
      place_id: row.id,
      inputs: {
        name: row.name,
        latitude: parseFloat(row.lat),
        longitude: parseFloat(row.lng),
        address: row.address
      }
    });
  }
  
  // Create batch
  const batch = await createBatch({
    batch_name: `CSV Import - ${filepath}`,
    places
  });
  
  return batch;
}

Progressive Results Download

async function downloadResults(batchId) {
  let offset = 0;
  const limit = 100;
  const allResults = [];
  
  while (true) {
    const response = await fetch(
      `https://reprompt--reprompt-fastapi-fastapi-app.us-west.modal.run/{org_slug}/place_enrichment/batches/${batchId}?offset=${offset}&limit=${limit}`,
      {
        headers: { 'Authorization': 'Bearer YOUR_API_KEY' }
      }
    );
    
    const data = await response.json();
    allResults.push(...data.results);
    
    if (data.results.length < limit) break;
    offset += limit;
  }
  
  return allResults;
}

Next Steps

Learn about Place Enrichment for single-item enrichment
Explore Geofencing for location-based batch queries
Check the API Reference for complete endpoint documentation

API Reference

Guides

​Overview

​Key Benefits

​Quick Start

​Batch Limits

​Batch Status Lifecycle

​Monitoring Progress

​Polling for Updates

​Progress Indicators

​Handling Results

​Success Results

​Failed Items

​Reprocessing Failed Items

​List All Batches

​Best Practices

​Common Patterns

​CSV Upload Processing

​Progressive Results Download

​Next Steps

Overview

Key Benefits

Quick Start

Batch Limits

Batch Status Lifecycle

Monitoring Progress

Polling for Updates

Progress Indicators

Handling Results

Success Results

Failed Items

Reprocessing Failed Items

List All Batches

Best Practices

Common Patterns

CSV Upload Processing

Progressive Results Download

Next Steps