SFTP
The SFTP
class allows you to interact with SFTP services,
using the Paramiko SFTP library under the hood.
The class provides methods to:
Create SFTP connections
Make, remove, and list the contents of directories
Get, put, remove, and check the size of files
Note
- Authentication
Depending on the server provider, SFTP may require either password or public key authentication. The
SFTP
class supports both methods viapassword
andrsa_private_key_file
arguments.
Quickstart
To instantiate SFTP
, pass your host name, user name, and either a password or an authentication
key file as keyword arguments:
from parsons import SFTP
sftp = SFTP(host='my_hostname', username='my_username', password='my_password')
# List contents of a directory
sftp.list_directory(remote_path='my_dir')
# Get a file
sftp.get_file(remote_path='my_dir/my_csv.csv', local_path='my_local_path/my_csv.csv')
To batch multiple methods using a single connection, you can create a connection and use
it in a with
block:
connection = sftp.create_connection()
with connection as conn:
sftp.make_directory('my_dir', connection=conn)
sftp.put_file('my_csv.csv', connection=conn)
API
- class parsons.SFTP(host, username, password, port=22, rsa_private_key_file=None, timeout: Optional[int] = None)[source]
Instantiate SFTP Class
- Args:
- host: str
The host name
- username: str
The user name
- password: str
The password
- rsa_private_key_file str
Absolute path to a private RSA key used to authenticate stfp connection
- port: int
Specify if different than the standard port 22
- timeout: int
Timeout argument for use when getting files through SFTP.
- Returns:
SFTP Class
- list_directory(remote_path='.', connection=None)[source]
List the contents of a directory
- Args:
- remote_path: str
The remote path of the directory
- connection: obj
An SFTP connection object
- Returns:
list of files and subdirectories in the provided directory
- make_directory(remote_path, connection=None)[source]
Makes a new directory on the SFTP server
- Args:
- remote_path: str
The remote path of the directory
- connection: obj
An SFTP connection object
- remove_directory(remote_path, connection=None)[source]
Remove a directory from the SFTP server
- Args:
- remote_path: str
The remote path of the directory
- connection: obj
An SFTP connection object
- get_file(remote_path, local_path=None, connection=None)[source]
Download a file from the SFTP server
- Args:
- remote_path: str
The remote path of the file to download
- local_path: str
The local path where the file will be downloaded. If not specified, a temporary file will be created and returned, and that file will be removed automatically when the script is done running.
- connection: obj
An SFTP connection object
- Returns:
- str
The path of the local file
- get_files(files_to_download=None, remote=None, connection=None, pattern=None, local_paths=None)[source]
Download a list of files, either by providing the list explicitly, providing directories that contain files to download, or both.
- Args:
- files_to_download: list
A list of full remote paths (can be relative) to files to download
- remote: str or list
A path to a remote directory or a list of paths
- connection: obj
An SFTP connection object
- pattern: str
A regex pattern with which to select file names. Defaults to None, in which case all files will be selected.
- local_paths: list
A list of paths to which to save the selected files. Defaults to None. If it is not the same length as the files to be fetched, temporary files are used instead.
- Returns:
- list
Local paths where the files are saved.
- get_table(remote_path, connection=None)[source]
Download a csv from the server and convert into a Parsons table.
The file may be compressed with gzip, or zip, but may not contain multiple files in the archive.
- Args:
- remote_path: str
The remote path of the file to download
- connection: obj
An SFTP connection object
- Returns:
- Parsons Table
See Parsons Table for output options.
- put_file(local_path: str, remote_path: str, connection=None, verbose: bool = True) None [source]
Put a file on the SFTP server
- Args:
- local_path: str
The local path of the source file
- remote_path: str
The remote path of the new file
- connection: obj
An SFTP connection object
- verbose: bool
Log progress every 5MB. Defaults to True.
- remove_file(remote_path, connection=None)[source]
Delete a file on the SFTP server
- Args:
- remote_path: str
The remote path of the file
- connection: obj
An SFTP connection object
- get_file_size(remote_path, connection=None)[source]
Get the size of a file in MB on the SFTP server. The file is not downloaded locally.
- Args:
- remote_path: str
The remote path of the file
- connection: obj
An SFTP connection object
- Returns:
- int
The file size in MB.
- list_subdirectories(remote_path, connection=None, pattern=None)[source]
List the subdirectories of a directory on the remote server.
- Args:
- remote_path: str
The remote directory whose subdirectories will be listed
- connection: obj
An SFTP connection object
- pattern: str
A regex pattern with which to select full directory paths. Defaults to None, in which case all subdirectories will be selected.
- Returns:
- list
The subdirectories in remote_path.
- list_files(remote_path, connection=None, pattern=None)[source]
List the files in a directory on the remote server.
- Args:
- remote_path: str
The remote directory whose files will be listed
- connection: obj
An SFTP connection object
- pattern: str
A regex pattern with which to select file names. Defaults to None, in which case all files will be selected.
- Returns:
- list
The files in remote_path.
- walk_tree(remote_path, connection=None, download=False, dir_pattern=None, file_pattern=None, max_depth=2)[source]
Recursively walks a directory, fetching all subdirectories and files (as long as they match dir_pattern and file_pattern, respectively) and the maximum directory depth hasn’t been exceeded. Optionally downloads discovered files.
- Args:
- remote_path: str
The top level directory to walk
- connection: obj
An SFTP connection object
- download: bool
Whether to download discovered files
- dir_pattern: str
A regex pattern with which to select directories. Defaults to None, in which case all directories will be selected.
- file_pattern: str
A regex pattern with which to select files. Defaults to None, in which case all files will be selected.
- max_depth: int
A limit on how many directories deep to traverse. The default, 2, will search the contents of remote_path and its subdirectories.
- Returns:
- tuple
A list of directories touched and a list of files. If the files were downloaded the file list will consist of local paths, if not, remote paths.