Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.octopipe.com/llms.txt

Use this file to discover all available pages before exploring further.

​
Using the Octopipe CLI Tutorial

This tutorial is designed to help you become proficient with the Octopipe Command-Line Interface (CLI). You’ll learn about the various commands available and how to use them effectively to manage your pipelines.

​
Introduction

Octopipe’s CLI offers a rich set of commands to initialize projects, manage data sources and destinations, define transformations, and orchestrate pipelines. In this guide, we will cover:
  • Basic CLI navigation
  • Essential commands and their usage
  • Tips and best practices for effective CLI usage

​
Getting Started with the CLI

​
Verifying Installation

After installing Octopipe, check that the CLI is working by running: 
octopipe --version
• Expected Output: The current version of Octopipe will be displayed. Accessing Help For general help, use: 
octopipe help
For command-specific help, append the command name. For example: 
octopipe help init
Core Commands Overview Initialization Initialize a new project with: 
octopipe init --name my_pipeline --description "Pipeline for sales data"
Authentication Log in to access Octopipe’s features: 
octopipe login --api-key YOUR_API_KEY
Log out when finished: 
octopipe logout
Managing Data Sources and Destinations Add a data source: 
octopipe source add --name sales_api --type api --option url=https://api.sales.com/data --option token=YOUR_TOKEN
Add a data destination: 
octopipe destination add --name sales_db --type postgres --option host=localhost --option port=5432 --option user=dbuser --option password=secret --option database=sales
Transformation and Pipeline Commands Define a transformation: 
octopipe transform add --name sales_transform --source sales_api --destination sales_db --schema-file ./schemas/sales_schema.json
Create a pipeline: 
octopipe pipeline create --name daily_sales --source sales_api --destination sales_db --transform sales_transform --schedule "0 0 * * *"
Start the pipeline: 
octopipe start daily_sales
Monitor logs: 
octopipe logs daily_sales --follow
Advanced CLI Usage • Verbose Mode: Add —verbose to commands for detailed output. For example: 
octopipe init --name verbose_pipeline --verbose
• Custom Configuration Files: Use the —config-file flag to load custom settings: 
octopipe init --name custom_pipeline --config-file ./config/custom.json
• Chaining Commands: Combine commands in scripts to automate tasks like initialization and deployment. Best Practices • Keep Commands Modular: Test each command individually to verify connectivity and correct configuration. • Use Descriptive Names: This helps maintain clarity when managing multiple pipelines. • Regularly Update the CLI: Stay current with the latest features and bug fixes. Troubleshooting CLI Issues • Command Not Found: Verify your PATH environment variable includes the Octopipe binary. • Authentication Errors: Double-check your API key or username/password if login fails. • Verbose Logging: Use the —verbose flag to get more detailed error messages. Conclusion Mastering the Octopipe CLI is key to efficiently managing your data pipelines. This tutorial provided an overview of essential commands and best practices. Experiment with these commands and refer to our detailed reference documents for more advanced usage. Happy CLI-ing!