# Ingest data (/docs/guides/ingest/ingest-data)



Ingest data [#ingest-data]

This guide uses the insert API in RawTree Cloud or RawTree BYOC.
The service handles credentials, table creation, and preprocessing. RawTree parses
and stores the resulting rows.

RawTree accepts unstructured data without a schema step. The service can create a table on first insert when your credential permits it.

Inline data [#inline-data]

```bash
rtree insert --table events \
  --data '{"action":"signup","user_id":1}'
```

Use an array to insert multiple rows:

```bash
rtree insert --table events \
  --data '[{"action":"signup","user_id":1},{"action":"purchase","user_id":1,"amount":42}]'
```

Files [#files]

```bash
rtree insert --table events --file ./events.jsonl
```

JSONL is useful for large event streams because each line is one event.

Public URLs [#public-urls]

```bash
rtree insert --table events --url https://example.com/events.jsonl
```

URL ingest waits for the import to finish. The API returns JSON with `inserted` (or `null` if the row count is unavailable). The `X-ClickHouse-Query-Id` response header contains the supplied ID, or the ID generated by RawTree when omitted. Errors return a normal HTTP error response.

Built-in transforms [#built-in-transforms]

Use transforms when the source format needs to be flattened before insert. A transform is a built-in preprocessor for JSON body inserts: the service reads the source JSON, emits one or more flat rows, then inserts those rows into the table.

```bash
rtree insert --table traces \
  --data '{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"api"}}]},"scopeSpans":[{"spans":[{"name":"GET /health","spanId":"abc"}]}]}' \
  --transform otlp-traces
```

The API accepts the same transform through the `transform` query parameter on a JSON body insert:

```bash test
curl -X POST "https://api.rawtree.com/v1/tables/traces?transform=otlp-traces" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"api"}}]},"scopeSpans":[{"spans":[{"name":"GET /health","spanId":"abc"}]}]}'
```

OpenTelemetry SDKs and collectors can also send native OTLP to the service without a `transform` query parameter. See the [OpenTelemetry guide](/docs/integrations/opentelemetry) for endpoints, SDK environment variables, Collector config, and a smoke test.

Transforms are not supported with URL inserts. If you use `?url=`, transform the data before hosting it.

See [Transforms](/docs/guides/transform/transforms) for supported input shapes and emitted rows.

| Transform         | Source format                                         |
| ----------------- | ----------------------------------------------------- |
| `otlp-traces`     | OpenTelemetry trace payloads                          |
| `otlp-logs`       | OpenTelemetry log payloads                            |
| `otlp-metrics`    | OpenTelemetry metric payloads                         |
| `cloudwatch-logs` | AWS CloudWatch Logs subscription or delivery payloads |
| `cloudtrail`      | AWS CloudTrail records                                |
| `firehose`        | AWS Firehose HTTP endpoint delivery payloads          |

API insert [#api-insert]

```bash test
curl -X POST "https://api.rawtree.com/v1/tables/events" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '[{"action":"signup","user_id":1}]'
```

Verify the insert [#verify-the-insert]

```bash
rtree table describe events
rtree query "SELECT count() FROM events"
```

Upload failures and retries [#upload-failures-and-retries]

Raw JSON inserts without a transform stream to the database. The service prepares a
missing table before it forwards the upload. It does not keep a complete copy for
server-side replay. Gzip uploads follow the same rules. The body-size limit
applies to decoded data.

Keep the full payload until the request succeeds. For temporary delivery
failures, including a table that is not visible on the destination replica, the
API returns `503` with `Retry-After`. Retry the full upload after that delay.
Use backoff if failures continue. Correct invalid JSON, invalid gzip, oversized
payloads, and permission errors before you retry.

A failed or interrupted upload can leave a table or some inserted rows. The
database can also accept data before a timeout or lost response. A retry can
therefore insert duplicate rows. A failed response does not guarantee rollback.