One common operation when it comes to working with HTTP is the ability to send up files. This is often done within the web browser, such as uploading a profile picture, or storing a file on a remote server, such as a video file. However, oftentimes when it comes to CLI applications, the ability to upload files is needed as well.

Attempting a Basic Approach

Currently, we know how to send data up to the HTTP bin service, through the use of a POST request. We've managed to achieve this by encoding our data in various different ways, such as as an HTTP form, or by encoding it through the use of, say, a JSON encoder. So let's go ahead and actually try and send up a file through this approach.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "os"
    "os/signal"
)

func main() {
    // Create a context that cancels on interrupt signal
    ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
    defer cancel()
    
    // Open the file we want to send
    file, _ := os.Open("words.txt")
    defer file.Close()
    
    // Create a request with the file as the body
    req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://httpbin.org/post", file)
    if err != nil {
        // Handle error
    }
    
    // Send the request
    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        // Handle error
    }
    defer resp.Body.Close()
    
    // Output the response
    io.Copy(os.Stdout, resp.Body)
}

Unfortunately, this approach doesn't work as expected. If we run this code, we'll see that the file content goes into the data field in the response, not the files field. This isn't what we want - web servers typically handle file uploads differently from regular form data.

Understanding Multipart Form Data

To properly upload files via HTTP, we need to use what's called a "multipart/form-data" request. This format:

• Allows mixing different data types (text, images, JSON) in a single request
• Properly handles binary data (essential for non-text files)
• Is the standard way web browsers upload files

The multipart format uses a special "boundary" string to separate different parts of the data being sent. Each part can have its own headers and content.

Using the multipart Package

Go's standard library provides the mime/multipart package which makes it easy to create multipart form data:

package main

import (
    "bytes"
    "context"
    "fmt"
    "io"
    "mime/multipart"
    "net/http"
    "os"
    "os/signal"
)

func main() {
    // Create a context that cancels on interrupt signal
    ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
    defer cancel()
    
    // Open the file we want to send
    file, _ := os.Open("words.txt")
    defer file.Close()
    
    // Create a buffer to store the multipart form data
    var buffer bytes.Buffer
    
    // Create a new multipart writer
    writer := multipart.NewWriter(&buffer)
    
    // Create a form file field
    formFile, _ := writer.CreateFormFile("words", "words.txt")
    
    // Copy the file content to the form field
    io.Copy(formFile, file)
    
    // Close the multipart writer to finalize it
    writer.Close()
    
    // Create a request with our multipart form data
    req, _ := http.NewRequestWithContext(ctx, http.MethodPost, "https://httpbin.org/post", &buffer)
    
    // Set the content type header with the multipart boundary
    req.Header.Set("Content-Type", writer.FormDataContentType())
    
    // Send the request
    resp, _ := http.DefaultClient.Do(req)
    defer resp.Body.Close()
    
    // Output the response
    io.Copy(os.Stdout, resp.Body)
}

Key components:

1. Buffer: We create a bytes.Buffer to store the multipart form data
2. Multipart Writer: The multipart.NewWriter creates a writer that handles the multipart formatting
3. Form File: CreateFormFile creates a section for a file upload with a form field name and filename
4. Content Type Header: We set the Content-Type header using writer.FormDataContentType(), which includes the boundary information

Running this code, the file will now appear in the "files" section of the response from httpbin.org.

Adding Form Fields

We can also add regular form fields alongside our file:

// After creating the file part but before closing the writer
formField, _ := writer.CreateFormField("username")
fmt.Fprint(formField, "Cloud")

With this, our username field will appear in the form values section of the response.

Complete File Upload Example

Here's a version with error handling included:

package main

import (
    "bytes"
    "context"
    "fmt"
    "io"
    "log"
    "mime/multipart"
    "net/http"
    "os"
    "os/signal"
    "path/filepath"
)

func main() {
    // Create a context that cancels on interrupt signal
    ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
    defer cancel()
    
    // Open the file we want to send
    file, err := os.Open("words.txt")
    if err != nil {
        log.Fatal("Failed to open file:", err)
    }
    defer file.Close()
    
    // Create a buffer to store the multipart form data
    var buffer bytes.Buffer
    
    // Create a new multipart writer
    writer := multipart.NewWriter(&buffer)
    
    // Create a form file field
    formFile, err := writer.CreateFormFile("words", "words.txt")
    if err != nil {
        log.Fatal("Failed to create form file:", err)
    }
    
    // Copy the file content to the form field
    _, err = io.Copy(formFile, file)
    if err != nil {
        log.Fatal("Failed to copy file:", err)
    }
    
    // Add a form field
    formField, err := writer.CreateFormField("username")
    if err != nil {
        log.Fatal("Failed to create form field:", err)
    }
    fmt.Fprint(formField, "Cloud")
    
    // Close the multipart writer to finalize it
    err = writer.Close()
    if err != nil {
        log.Fatal("Failed to close multipart writer:", err)
    }
    
    // Create a request with our multipart form data
    req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://httpbin.org/post", &buffer)
    if err != nil {
        log.Fatal("Failed to create request:", err)
    }
    
    // Set the content type header with the multipart boundary
    req.Header.Set("Content-Type", writer.FormDataContentType())
    
    // Send the request
    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        log.Fatal("Failed to send request:", err)
    }
    defer resp.Body.Close()
    
    // Output the response
    io.Copy(os.Stdout, resp.Body)
}

Go makes it incredibly easy to be able to perform HTTP requests and the various different types of requests that you need to use, especially when it comes to sending up things such as form data, JSON encoding, or even files themselves.