SFTP

The SFTP class allows you to interact with SFTP services, using the Paramiko SFTP library under the hood.

The class provides methods to:

  • Create SFTP connections
  • Make, remove, and list the contents of directories
  • Get, put, remove, and check the size of files

Note

Authentication
Depending on the server provider, SFTP may require either password or public key authentication. The SFTP class supports both methods via password and rsa_private_key_file arguments.

Quickstart

To instantiate SFTP, pass your host name, user name, and either a password or an authentication key file as keyword arguments:

from parsons import SFTP

sftp = SFTP(host='my_hostname', username='my_username', password='my_password')

# List contents of a directory
sftp.list_directory(remote_path='my_dir')

# Get a file
sftp.get_file(remote_path='my_dir/my_csv.csv', local_path='my_local_path/my_csv.csv')

To batch multiple methods using a single connection, you can create a connection and use it in a with block:

connection = sftp.create_connection()

with connection as conn:
    sftp.make_directory('my_dir', connection=conn)
    sftp.put_file('my_csv.csv', connection=conn)

API

class parsons.SFTP(host, username, password, port=22, rsa_private_key_file=None)[source]

Instantiate SFTP Class

Args:
host: str
The host name
username: str
The user name
password: str
The password
rsa_private_key_file str
Absolute path to a private RSA key used to authenticate stfp connection
port: int
Specify if different than the standard port 22
Returns:
SFTP Class
create_connection()[source]

Create an SFTP connection.

Returns:
SFTP Connection object
list_directory(remote_path='.', connection=None)[source]

List the contents of a directory

Args:
remote_path: str
The remote path of the directory
connection: obj
An SFTP connection object
Returns:
list of files and subdirectories in the provided directory
make_directory(remote_path, connection=None)[source]

Makes a new directory on the SFTP server

Args:
remote_path: str
The remote path of the directory
connection: obj
An SFTP connection object
remove_directory(remote_path, connection=None)[source]

Remove a directory from the SFTP server

Args:
remote_path: str
The remote path of the directory
connection: obj
An SFTP connection object
get_file(remote_path, local_path=None, connection=None)[source]

Download a file from the SFTP server

Args:
remote_path: str
The remote path of the file to download
local_path: str
The local path where the file will be downloaded. If not specified, a temporary file will be created and returned, and that file will be removed automatically when the script is done running.
connection: obj
An SFTP connection object
Returns:
str
The path of the local file
get_files(files_to_download=None, remote=None, connection=None, pattern=None, local_paths=None)[source]

Download a list of files, either by providing the list explicitly, providing directories that contain files to download, or both.

Args:
files_to_download: list
A list of full remote paths (can be relative) to files to download
remote: str or list
A path to a remote directory or a list of paths
connection: obj
An SFTP connection object
pattern: str
A regex pattern with which to select file names. Defaults to None, in which case all files will be selected.
local_paths: list
A list of paths to which to save the selected files. Defaults to None. If it is not the same length as the files to be fetched, temporary files are used instead.
Returns:
list
Local paths where the files are saved.
get_table(remote_path, connection=None)[source]

Download a csv from the server and convert into a Parsons table.

The file may be compressed with gzip, or zip, but may not contain multiple files in the archive.

Args:
remote_path: str
The remote path of the file to download
connection: obj
An SFTP connection object
Returns:
Parsons Table
See Parsons Table for output options.
put_file(local_path, remote_path, connection=None)[source]

Put a file on the SFTP server

Args:
local_path: str
The local path of the source file
remote_path: str
The remote path of the new file
connection: obj
An SFTP connection object
remove_file(remote_path, connection=None)[source]

Delete a file on the SFTP server

Args:
remote_path: str
The remote path of the file
connection: obj
An SFTP connection object
get_file_size(remote_path, connection=None)[source]

Get the size of a file in MB on the SFTP server. The file is not downloaded locally.

Args:
remote_path: str
The remote path of the file
connection: obj
An SFTP connection object
Returns:
int
The file size in MB.
list_subdirectories(remote_path, connection=None, pattern=None)[source]

List the subdirectories of a directory on the remote server.

Args:
remote_path: str
The remote directory whose subdirectories will be listed
connection: obj
An SFTP connection object
pattern: str
A regex pattern with which to select full directory paths. Defaults to None, in which case all subdirectories will be selected.
Returns:
list
The subdirectories in remote_path.
list_files(remote_path, connection=None, pattern=None)[source]

List the files in a directory on the remote server.

Args:
remote_path: str
The remote directory whose files will be listed
connection: obj
An SFTP connection object
pattern: str
A regex pattern with which to select file names. Defaults to None, in which case all files will be selected.
Returns:
list
The files in remote_path.
walk_tree(remote_path, connection=None, download=False, dir_pattern=None, file_pattern=None, max_depth=2)[source]

Recursively walks a directory, fetching all subdirectories and files (as long as they match dir_pattern and file_pattern, respectively) and the maximum directory depth hasn’t been exceeded. Optionally downloads discovered files.

Args:
remote_path: str
The top level directory to walk
connection: obj
An SFTP connection object
download: bool
Whether to download discovered files
dir_pattern: str
A regex pattern with which to select directories. Defaults to None, in which case all directories will be selected.
file_pattern: str
A regex pattern with which to select files. Defaults to None, in which case all files will be selected.
max_depth: int
A limit on how many directories deep to traverse. The default, 2, will search the contents of remote_path and its subdirectories.
Returns:
tuple
A list of directories touched and a list of files. If the files were downloaded the file list will consist of local paths, if not, remote paths.