​

octopipe init Command Reference

The octopipe init command is your starting point for creating a new Octopipe project or pipeline. It sets up the initial directory structure and configuration files required for the pipeline to function correctly.

​

Purpose

• Project Setup: Initializes a new project by creating a working directory with the necessary configuration.
• Configuration Initialization: Loads default settings and prepares the environment for subsequent commands.

​

Usage

octopipe init [options]

Options

• --name <pipeline_name>

Description: Sets the unique name for your project or pipeline.

Example: —name my_pipeline

• --description <description>

Description: Provides a short description of the project for documentation purposes.

Example: —description “ETL pipeline for sales data”

• —local

Description: Initializes the project for local development.

Example: Use this flag when planning to run pipelines on your own machine.

• —cloud

Description: Sets up the project for cloud deployment. This is the default if no environment flag is specified.

Example: Use when deploying pipelines on Octopipe’s cloud infrastructure.

• --config-file <path>

Description: Specifies a custom configuration file (in JSON or YAML format) to initialize the project with custom settings.

Example: —config-file ./config/settings.json

Detailed Behavior

• Directory Creation:

The command creates a structured project directory with subdirectories for source code, configuration files, and logs. This helps maintain consistency across projects.

• Configuration Files:

Default configuration files are generated, which can later be edited to tailor the pipeline to your needs. This includes settings for API keys, default models, and other environmental variables.

• Interactive Prompts:

In some cases, if parameters are not provided via options, the CLI will prompt you interactively to fill in missing details.

Examples

Example 1: Local Project Initialization

octopipe init --name local_etl --description "Local ETL pipeline for testing" --local

Example 2: Cloud-Ready Project with Custom Config

octopipe init --name cloud_pipeline --description "Pipeline for production use" --config-file ./config/prod_settings.yaml

Best Practices

• Naming Conventions:

Choose descriptive and unique names for your pipelines to avoid conflicts.

• Documentation:

Always include a description to help future maintainers understand the project’s purpose.

• Configuration Management:

Consider using version-controlled configuration files to track changes over time.

Troubleshooting

• Missing Options:

If required options are missing, the CLI will prompt you interactively. Make sure to review the prompts carefully.

• Directory Permissions:

Ensure you have write permissions in the directory where you are initializing the project.

• Configuration File Errors:

If a custom configuration file is provided and fails to load, check the file format and validate against the schema provided in our documentation.

Conclusion

The octopipe init command lays the foundation for your data pipeline project. By using it correctly, you ensure that your project starts with the correct structure and configuration, paving the way for smooth development and deployment.