Automate File Transfers with SFTP in Bash

Published {$created} by Viggo

As a developer, you often need to automate file transfers. While graphical FTP clients are great for occasional use, scripting repetitive tasks demands a command-line solution. This tutorial focuses on how to use SFTP (Secure FTP) in Bash for robust and secure file transfers. We're leveraging SFTP over regular FTP because of its encryption, a key security consideration. For a deeper dive, check out FTP 101: SFTP vs. FTP vs. FTPS.

Why SFTP in Bash?

Bash scripting provides a powerful way to automate tasks within a Linux or macOS environment. Combined with SFTP, you can:

• Automate Backups: Easily create scripts to backup important files or directories to your ftpGrid account. Consider this alternative to self-hosted solutions – explore our backup storage options.
• Synchronize Files: Keep files in sync between your local machine and a remote server.
• Integrate with CI/CD Pipelines: Automate deployment of code or other assets.
• Create Custom Tools: Build custom file management utilities.

Basic SFTP Commands in Bash

The sftp command is your primary tool. Here are some common commands:

• sftp user@edgeN.ftpgrid.com: Connects to the server. Replace edgeN.ftpgrid.com with your actual hostname.
• get remote_file: Downloads a file from the server.
• put local_file: Uploads a file to the server.
• ls: Lists files on the server.
• cd directory: Changes the current directory on the server.
• mkdir directory: Creates a directory on the server.
• rm file: Removes a file on the server.
• exit: Disconnects from the server.

Example Bash Script

Here's a simple script to upload a directory:

#!/bin/bash

# Configuration
SFTP_USER="your_username"
SFTP_HOST="edgeN.ftpgrid.com"
LOCAL_DIR="./local_files"
REMOTE_DIR="/your_remote_directory"

# Create SSH keypair if doesn't exist
if [ ! -f ~/.ssh/id_rsa.pub ]; then
  echo "Creating SSH keypair.  This is required for authentication. See [Create SSH keys for SFTP, SCP authentication](https://ftpgrid.com/tutorials/create-ssh-keys-for-sftp-scp-authentication/)"
  ssh-keygen -t rsa -b 2048
fi

# SFTP command
sftp -o "StrictHostKeyChecking=no" $SFTP_USER@$SFTP_HOST << EOF
lcd $LOCAL_DIR
cd $REMOTE_DIR
mput *
exit
EOF

echo "Files uploaded successfully."

Important:

• Replace placeholders with your actual credentials.
• This script uses StrictHostKeyChecking=no. While convenient for initial setup, understand the security implications and consider adding the host key to your known_hosts file for better security. You can automate this with ssh-keyscan if you want to be completely hands-off.
• This script assumes passwordless login using SSH keys. If you're using password authentication (not recommended!), you'll need to explore alternatives like prompting for the password within the script or using expect. For secure authentication, check out Advanced SSH Key Authentication for SFTP.

Key Considerations

• SSH Keys: Always prefer SSH keys over password authentication. They’re significantly more secure.
• Error Handling: Add error handling to your scripts to gracefully handle failures.
• Logging: Log important events for debugging and auditing.
• Security: Be mindful of security best practices. Avoid hardcoding credentials in your scripts. Consider using environment variables or a secure configuration file. If you are uncomfortable setting up keys yourself, consider a managed solution like ftpGrid, which handles all this for you!

This tutorial provides a basic introduction to SFTP in Bash. With a little practice, you can create powerful scripts to automate your file transfer needs. For more in-depth information, explore our Quick Storage API series and the tutorials. If you are looking for a hassle-free solution with robust features like quota management and audit logging, consider ftpGrid.