Command Line Applications in Go

Locked video

Please purchase the course to watch this video.

Powerful Command Line Applications

Environment variables

Environment variables provide a flexible and secure way to configure CLI applications, enabling developers to avoid hard-coding sensitive data such as database URLs, API tokens, or AWS credentials. In Go, environment variables can be accessed using the os.Getenv function, which returns an empty string if the variable doesn't exist, or os.LookupEnv, which additionally indicates whether the variable was set. Managing environment variables during development is often streamlined by using a .env file to specify key-value pairs, and these can be loaded into the application either via popular packages like go.env or through custom loaders that read the file, safely split keys and values, trim any quotes, and set variables using os.Setenv. It's important to never commit .env files to version control, and variables can be unset when no longer needed. Understanding how to handle environment variables directly gives developers greater insight into application configuration best practices and security.

No links available for this lesson.

In this lesson, we're going to talk about environment variables, which are a common way to configure CLI applications without needing to hard code values or constantly pass data to them as flags.

We've actually seen environment variables already, specifically when it came to our $EDITOR environment variable:

bash

echo $EDITOR
# Output: nvim (or your configured default editor)

When we were building the mechanism to open up our text editor with some data, we actually pulled out this editor value using the os.Getenv() function.

Common Use Cases

Environment variables are particularly useful for several scenarios:

1. Configuration Without Hard-coding

Instead of embedding configuration directly in code, use environment variables for flexibility.

2. Secrets Management

One of the more common uses is for pulling out secrets:

bash

# Database credentials
export DATABASE_URL="postgres://myuser:mypassword@mypostgreshost:5432/mydb"

# API tokens
export API_TOKEN="your-secret-api-token"
export API_KEY="your-api-key"

# AWS credentials
export AWS_PROFILE="production"
export AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

Then in your application:

dbURL := os.Getenv("DATABASE_URL")
apiToken := os.Getenv("API_TOKEN")

Two Functions for Reading Environment Variables

Go provides two different functions for pulling out environment variables:

1. `os.Getenv()` - Simple Retrieval

package main

import (
    "fmt"
    "os"
)

func main() {
    dbURL := os.Getenv("DATABASE_URL")
    fmt.Println("Database URL:", dbURL)
}

Behavior: Returns the value, or an empty string if the variable is not present.

2. `os.LookupEnv()` - With Existence Check

package main

import (
    "fmt"
    "os"
)

func main() {
    value, exists := os.LookupEnv("NO_EXIST_ENV_VAR")
    
    fmt.Printf("Value: '%s'\tExists: %t\n", value, exists)
    
    if !exists {
        fmt.Println("Please set NO_EXIST_ENV_VAR environment variable")
        return
    }
    
    fmt.Println("Environment variable value:", value)
}

Behavior: Returns the value and a boolean indicating whether the variable was set.

When to Use Which?

os.LookupEnv() is generally preferred because it allows you to distinguish between:

Variable not set at all
Variable set to an empty string

This distinction is important for proper error handling and configuration validation.

Testing the Difference

bash

# Run without setting the variable
go run main.go
# Output: Value: ''    Exists: false
#         Please set NO_EXIST_ENV_VAR environment variable

# Run with the variable set inline
NO_EXIST_ENV_VAR="hello world" go run main.go
# Output: Value: 'hello world'    Exists: true
#         Environment variable value: hello world

Working with .env Files

Often when developing code, setting environment variables manually can be tedious. A common pattern is to define a .env file where you specify all the environment variables you need:

Creating a .env File

bash

# .env file content
DATABASE_URL="postgres://user:pass@localhost:5432/mydb"
API_TOKEN="my-secret-api-token"
SECRET_KEY="super-secret-key-with=equals-sign"

Important: .env File Security

⚠️ Critical: .env files should never be committed to version control!

bash

# Initialize git and check status
git init
git status
# .env will appear in untracked files

# Add to .gitignore
echo ".env" >> .gitignore
git status
# .env should no longer appear in untracked files

Building a Custom .env Loader

While the godotenv package is commonly used for this, let's implement our own simple .env loader to understand how it works under the hood.

Implementation

package main

import (
    "bufio"
    "fmt"
    "os"
    "strings"
)

func loadEnvironment() error {
    // Open the .env file
    file, err := os.Open(".env")
    if err != nil {
        return err
    }
    defer file.Close()
    
    // Create scanner to read line by line
    scanner := bufio.NewScanner(file)
    
    for scanner.Scan() {
        line := scanner.Text()
        
        // Skip empty lines and comments
        if len(line) == 0 || strings.HasPrefix(line, "#") {
            continue
        }
        
        // Split on equals sign (limit to 2 parts)
        parts := strings.SplitN(line, "=", 2)
        
        // Validate line format
        if len(parts) < 2 {
            continue // Invalid line, skip
        }
        
        key := parts[0]
        value := strings.Trim(parts[1], `"`) // Remove quotes
        
        // Set environment variable
        err := os.Setenv(key, value)
        if err != nil {
            return err
        }
        
        fmt.Printf("Loaded: %s = %s\n", key, value)
    }
    
    return scanner.Err()
}

func main() {
    // Load environment variables from .env file
    if err := loadEnvironment(); err != nil {
        fmt.Printf("Error loading .env file: %v\n", err)
        return
    }
    
    // Test accessing the loaded variables
    dbURL, exists := os.LookupEnv("DATABASE_URL")
    if !exists {
        fmt.Println("DATABASE_URL not set")
        return
    }
    
    fmt.Println("Database URL:", dbURL)
}

Why `strings.SplitN(line, "=", 2)`?

Using SplitN with a limit of 2 is crucial:

bash

# Without limit (using strings.Split):
SECRET_KEY="password=with=equals"
# Would split into: ["SECRET_KEY", "\"password", "with", "equals\""]

# With limit of 2 (using strings.SplitN):
SECRET_KEY="password=with=equals"  
# Splits into: ["SECRET_KEY", "\"password=with=equals\""]

This ensures we only split on the first equals sign, preserving any equals signs in the value.

Handling Quoted Values

The strings.Trim(parts[1], ") removes surrounding quotes:

// Before trim: "\"my-secret-value\""
// After trim:  "my-secret-value"

Complete Working Example

Here's a complete example demonstrating environment variable usage:

package main

import (
    "bufio"
    "fmt"
    "log"
    "os"
    "strings"
)

func loadEnvironment() error {
    file, err := os.Open(".env")
    if err != nil {
        // .env file is optional
        return nil
    }
    defer file.Close()
    
    scanner := bufio.NewScanner(file)
    for scanner.Scan() {
        line := strings.TrimSpace(scanner.Text())
        
        // Skip empty lines and comments
        if len(line) == 0 || strings.HasPrefix(line, "#") {
            continue
        }
        
        parts := strings.SplitN(line, "=", 2)
        if len(parts) != 2 {
            continue
        }
        
        key := strings.TrimSpace(parts[0])
        value := strings.Trim(strings.TrimSpace(parts[1]), `"'`)
        
        if err := os.Setenv(key, value); err != nil {
            return fmt.Errorf("setting %s: %w", key, err)
        }
    }
    
    return scanner.Err()
}

func getRequiredEnv(key string) string {
    value, exists := os.LookupEnv(key)
    if !exists {
        log.Fatalf("Required environment variable %s is not set", key)
    }
    return value
}

func getEnvWithDefault(key, defaultValue string) string {
    if value, exists := os.LookupEnv(key); exists {
        return value
    }
    return defaultValue
}

func main() {
    // Load .env file if it exists
    if err := loadEnvironment(); err != nil {
        log.Fatal("Error loading environment:", err)
    }
    
    // Required environment variables
    dbURL := getRequiredEnv("DATABASE_URL")
    apiToken := getRequiredEnv("API_TOKEN")
    
    // Optional environment variables with defaults
    port := getEnvWithDefault("PORT", "8080")
    logLevel := getEnvWithDefault("LOG_LEVEL", "info")
    
    fmt.Printf("Database URL: %s\n", dbURL)
    fmt.Printf("API Token: %s\n", apiToken)
    fmt.Printf("Port: %s\n", port)
    fmt.Printf("Log Level: %s\n", logLevel)
}

Sample .env File

bash

# Database configuration
DATABASE_URL="postgres://user:password@localhost:5432/myapp"

# API configuration  
API_TOKEN="secret-api-token-here"

# Optional settings
PORT="3000"
LOG_LEVEL="debug"

# Complex values with special characters
SECRET_KEY="my-secret=with=equals&special!chars"

Environment Variable Management

Setting Variables

bash

# Temporary (current session only)
export API_TOKEN="your-token-here"

# Inline for single command
API_TOKEN="your-token" go run main.go

# Using .env file (with our loader)
echo 'API_TOKEN="your-token"' > .env
go run main.go

Unsetting Variables

// In Go code
err := os.Unsetenv("API_TOKEN")
if err != nil {
    log.Fatal("Error unsetting variable:", err)
}

bash

# In shell
unset API_TOKEN

Checking Variables

bash

# Check if variable is set
echo $API_TOKEN

# List all environment variables
env

# List variables matching pattern
env | grep API

Best Practices

1. Use Descriptive Names

bash

# Good
DATABASE_URL="..."
API_TOKEN="..."
LOG_LEVEL="debug"

# Bad  
DB="..."
TOKEN="..."
LEVEL="debug"

2. Provide Defaults for Optional Settings

port := getEnvWithDefault("PORT", "8080")
timeout := getEnvWithDefault("TIMEOUT", "30s")

3. Validate Required Variables Early

func init() {
    requiredVars := []string{
        "DATABASE_URL",
        "API_TOKEN", 
        "SECRET_KEY",
    }
    
    for _, v := range requiredVars {
        if _, exists := os.LookupEnv(v); !exists {
            log.Fatalf("Required environment variable %s is not set", v)
        }
    }
}

4. Use Different .env Files for Different Environments

bash

.env.development
.env.staging  
.env.production
.env.test

5. Document Your Environment Variables

// Environment variables used by this application:
//
// Required:
//   DATABASE_URL - PostgreSQL connection string
//   API_TOKEN    - Authentication token for external API
//   SECRET_KEY   - Encryption key for sensitive data
//
// Optional:
//   PORT         - Server port (default: 8080)
//   LOG_LEVEL    - Logging level (default: info)
//   TIMEOUT      - Request timeout (default: 30s)

Production Considerations

1. Never Commit Secrets

gitignore

# .gitignore
.env
.env.local
.env.*.local
*.key
*.pem

2. Use Secret Management Systems

For production environments, consider:

AWS Secrets Manager
HashiCorp Vault
Kubernetes Secrets
Azure Key Vault

3. Environment Validation

func validateEnvironment() error {
    dbURL := os.Getenv("DATABASE_URL")
    if !strings.HasPrefix(dbURL, "postgres://") {
        return errors.New("DATABASE_URL must be a valid PostgreSQL connection string")
    }
    
    if len(os.Getenv("SECRET_KEY")) < 32 {
        return errors.New("SECRET_KEY must be at least 32 characters")
    }
    
    return nil
}

Summary

Environment variables provide a powerful way to configure CLI applications:

✅ Keep secrets out of code - Store sensitive data externally
✅ Environment-specific configuration - Different settings per environment
✅ No recompilation needed - Change behavior without rebuilding
✅ 12-Factor App compliance - Follow cloud-native principles
✅ Easy deployment - Simple configuration management

Key Functions:

os.Getenv() - Simple retrieval (returns empty string if not set)
os.LookupEnv() - Retrieval with existence check (preferred)
os.Setenv() - Set environment variables programmatically
os.Unsetenv() - Remove environment variables

What's Next?

In the next lesson, we're going to look at another approach for getting configuration into your application through the use of config files and the embed package.

🔒 Security Reminder

Always remember:

Never commit .env files to version control
Use different configuration for different environments
Validate environment variables at startup
Consider using dedicated secret management solutions for production

No homework tasks for this lesson.

How would you rate this lesson?

Full Course

Common Use Cases

1. Configuration Without Hard-coding

2. Secrets Management

Two Functions for Reading Environment Variables

1. `os.Getenv()` - Simple Retrieval

2. `os.LookupEnv()` - With Existence Check

When to Use Which?

Testing the Difference

Working with .env Files

Creating a .env File

Important: .env File Security

Building a Custom .env Loader

Implementation

Why `strings.SplitN(line, "=", 2)`?

Handling Quoted Values

Complete Working Example

Sample .env File

Environment Variable Management

Setting Variables

Unsetting Variables

Checking Variables

Best Practices

1. Use Descriptive Names

2. Provide Defaults for Optional Settings

3. Validate Required Variables Early

4. Use Different .env Files for Different Environments

5. Document Your Environment Variables

Production Considerations

1. Never Commit Secrets

2. Use Secret Management Systems

3. Environment Validation

Summary

What's Next?

🔒 Security Reminder

Get Support

Full Course

Common Use Cases

1. Configuration Without Hard-coding

2. Secrets Management

Two Functions for Reading Environment Variables

1. os.Getenv() - Simple Retrieval

2. os.LookupEnv() - With Existence Check

When to Use Which?

Testing the Difference

Working with .env Files

Creating a .env File

Important: .env File Security

Building a Custom .env Loader

Implementation

Why strings.SplitN(line, "=", 2)?

Handling Quoted Values

Complete Working Example

Sample .env File

Environment Variable Management

Setting Variables

Unsetting Variables

Checking Variables

Best Practices

1. Use Descriptive Names

2. Provide Defaults for Optional Settings

3. Validate Required Variables Early

4. Use Different .env Files for Different Environments

5. Document Your Environment Variables

Production Considerations

1. Never Commit Secrets

2. Use Secret Management Systems

3. Environment Validation

Summary

What's Next?

🔒 Security Reminder

Get Support

1. `os.Getenv()` - Simple Retrieval

2. `os.LookupEnv()` - With Existence Check

Why `strings.SplitN(line, "=", 2)`?