ReferenceAPIPipeline Service
Resume pipeline
POST
workspace*stringThe workspace id.
pipeline*stringThe pipeline id.
application/json- body
name?stringstream?stringA stream paused on its own; empty: the whole pipeline, which leaves paused streams paused.
OK
application/json- response
pipeline?curl -X POST 'http://localhost:8080/v1/workspaces/string/pipelines/string:resume' \ -H 'Content-Type: application/json' \ -d '{}'Example Response
{ "pipeline": { "name": "string", "spec": { "mode": "PIPELINE_MODE_UNSPECIFIED", "schedule": { "every": "string", "cron": "string", "jitter": "string" }, "dataPlane": "string", "source": { "ref": "string", "streams": [ { "name": "string", "sync": "SYNC_MODE_UNSPECIFIED", "cursor": "string", "columns": { "property1": { "type": "COLUMN_TYPE_UNSPECIFIED", "held": { "mode": "HELD_MODE_UNSPECIFIED", "value": "string" } }, "property2": { "type": "COLUMN_TYPE_UNSPECIFIED", "held": { "mode": "HELD_MODE_UNSPECIFIED", "value": "string" } } }, "timestamps": "TIMESTAMP_UNIT_UNSPECIFIED", "require": "DELIVERY_LEVEL_UNSPECIFIED", "collapse": true, "quarantine": "QUARANTINE_SCOPE_UNSPECIFIED", "held": { "mode": "HELD_MODE_UNSPECIFIED", "value": "string" }, "groups": [ { "columns": [ "string" ] } ], "typeChanges": "TYPE_CHANGES_UNSPECIFIED" } ] }, "destination": { "ref": "string", "write": "WRITE_MODE_UNSPECIFIED", "namespace": "string", "naming": "NAMING_UNSPECIFIED", "deletes": "DELETE_MODE_UNSPECIFIED", "typeChanges": "TYPE_CHANGES_UNSPECIFIED", "tables": { "property1": "string", "property2": "string" } }, "schemaPolicy": "SCHEMA_POLICY_UNSPECIFIED", "delivery": { "require": "DELIVERY_LEVEL_UNSPECIFIED", "commit": { "interval": "string", "maxBytes": "string", "scope": "COMMIT_SCOPE_UNSPECIFIED" }, "resumePointTimeout": "string" }, "types": { "infinity": "INFINITY_UNSPECIFIED" }, "postSync": [ { "dbt": { "project": "string", "command": "DBT_COMMAND_UNSPECIFIED", "select": "string", "exclude": "string", "schema": "string" }, "spark": { "cluster": "string", "code": { "jar": "string", "python": "string", "sha256": "string", "mainClass": "string" }, "args": [ "string" ], "conf": { "property1": "string", "property2": "string" } } } ], "heal": { "mode": "HEAL_MODE_UNSPECIFIED" }, "transform": { "spark": { "cluster": "string", "code": { "jar": "string", "python": "string", "sha256": "string", "mainClass": "string" }, "conf": { "property1": "string", "property2": "string" }, "outputs": [ { "name": "string", "primaryKey": [ "string" ], "write": "WRITE_MODE_UNSPECIFIED", "require": "DELIVERY_LEVEL_UNSPECIFIED" } ] }, "flink": { "cluster": "string", "code": { "jar": "string", "sha256": "string", "mainClass": "string" }, "parallelism": 0, "conf": { "property1": "string", "property2": "string" }, "outputs": [ { "name": "string", "primaryKey": [ "string" ], "write": "WRITE_MODE_UNSPECIFIED", "require": "DELIVERY_LEVEL_UNSPECIFIED" } ] } }, "sparkAccess": { "emrRole": "string", "keys": {} }, "flinkAccess": { "emrRole": "string", "keys": {} }, "resources": { "size": { "preset": "string", "cpuMillis": "string", "memoryBytes": "string" }, "min": { "preset": "string", "cpuMillis": "string", "memoryBytes": "string" }, "max": { "preset": "string", "cpuMillis": "string", "memoryBytes": "string" }, "finishWithin": "string", "pool": "string" } }, "version": "string", "etag": "string", "createTime": "2019-08-24T14:15:22Z", "updateTime": "2019-08-24T14:15:22Z", "paused": true, "status": { "lastSync": { "name": "string", "version": "string", "state": "SYNC_STATE_UNSPECIFIED", "error": "string", "startTime": "2019-08-24T14:15:22Z", "endTime": "2019-08-24T14:15:22Z", "dataPlane": "string", "streams": [ { "name": "string", "rows": "string", "quarantinedRows": "string", "collapsedRows": "string", "heldCells": "string", "heldColumns": [ "string" ] } ], "continuous": true, "lastCommitTime": "2019-08-24T14:15:22Z", "progress": [ { "stream": "string", "lagBytes": "string", "lagRecords": "string", "lag": "string", "retentionLeftBytes": "string", "retentionLeft": "string", "time": "2019-08-24T14:15:22Z" } ], "failedAttempts": 0, "lastAttemptError": "string", "lastAttemptTime": "2019-08-24T14:15:22Z", "attempts": [ { "attempt": 0, "size": { "preset": "string", "cpuMillis": "string", "memoryBytes": "string" }, "sizeReason": "JOB_SIZE_REASON_UNSPECIFIED", "usage": { "cpuSeconds": 0.1, "peakMemoryBytes": "string", "runnerCpuSeconds": 0.1, "runnerMemoryBytes": "string", "seconds": 0.1 }, "rows": "string", "bytes": "string", "code": "string", "lanes": { "lanes": 0, "limit": 0, "busySeconds": 0.1, "backpressuredSeconds": 0.1, "idleSeconds": 0.1, "rows": "string" } } ] }, "lastSuccessTime": "2019-08-24T14:15:22Z", "openIncidents": 0, "scaling": { "size": { "preset": "string", "cpuMillis": "string", "memoryBytes": "string" }, "lanes": 0, "rescaleTime": "2019-08-24T14:15:22Z", "note": "string" } }, "pausedStreams": [ { "stream": "string", "reason": "string", "incident": "string", "createTime": "2019-08-24T14:15:22Z" } ], "overrides": [ { "field": "string", "value": "string", "specValue": "string", "reason": "string", "incident": "string", "updateTime": "2019-08-24T14:15:22Z" } ] }}Reset pipeline POST
Reads streams again from the start: the next sync, or a continuous pipeline's next run, drops their positions and reads them as it would a new stream's, such as a new replication slot and a snapshot. What reaches the destination already stays: merge updates rows in place, and append adds them again. A continuous pipeline's run restarts at a checkpoint to take it. Fails with FAILED_PRECONDITION for a pipeline with a Flink transform, whose state Flink keeps.
Rewind pipeline POST
Lets the next sync or run restart from the checkpoint before the latest one if the latest can't be read. Its epoch was delivered, so it's delivered again: streams the destination neither merges nor dedupes get its rows twice. Without this, a restart happens by itself only where it can't lower a stream's guarantee, under heal mode auto or auto_safe.