SFTP¶
The SFTP
class allows you to interact with SFTP services,
using the Paramiko SFTP library under the hood.
The class provides methods to:
- Create SFTP connections
- Make, remove, and list the contents of directories
- Get, put, remove, and check the size of files
Note
- Authentication
- Depending on the server provider, SFTP may require either password or public key authentication.
The
SFTP
class supports both methods viapassword
andrsa_private_key_file
arguments.
Quickstart¶
To instantiate SFTP
, pass your host name, user name, and either a password or an authentication
key file as keyword arguments:
from parsons import SFTP
sftp = SFTP(host='my_hostname', username='my_username', password='my_password')
# List contents of a directory
sftp.list_directory(remote_path='my_dir')
# Get a file
sftp.get_file(remote_path='my_dir/my_csv.csv', local_path='my_local_path/my_csv.csv')
To batch multiple methods using a single connection, you can create a connection and use
it in a with
block:
connection = sftp.create_connection()
with connection as conn:
sftp.make_directory('my_dir', connection=conn)
sftp.put_file('my_csv.csv', connection=conn)
API¶
-
class
parsons.
SFTP
(host, username, password, port=22, rsa_private_key_file=None)[source]¶ Instantiate SFTP Class
- Args:
- host: str
- The host name
- username: str
- The user name
- password: str
- The password
- rsa_private_key_file str
- Absolute path to a private RSA key used to authenticate stfp connection
- port: int
- Specify if different than the standard port 22
- Returns:
- SFTP Class
-
list_directory
(remote_path='.', connection=None)[source]¶ List the contents of a directory
- Args:
- remote_path: str
- The remote path of the directory
- connection: obj
- An SFTP connection object
- Returns:
- list of files and subdirectories in the provided directory
-
make_directory
(remote_path, connection=None)[source]¶ Makes a new directory on the SFTP server
- Args:
- remote_path: str
- The remote path of the directory
- connection: obj
- An SFTP connection object
-
remove_directory
(remote_path, connection=None)[source]¶ Remove a directory from the SFTP server
- Args:
- remote_path: str
- The remote path of the directory
- connection: obj
- An SFTP connection object
-
get_file
(remote_path, local_path=None, connection=None)[source]¶ Download a file from the SFTP server
- Args:
- remote_path: str
- The remote path of the file to download
- local_path: str
- The local path where the file will be downloaded. If not specified, a temporary file will be created and returned, and that file will be removed automatically when the script is done running.
- connection: obj
- An SFTP connection object
- Returns:
- str
- The path of the local file
-
get_files
(files_to_download=None, remote=None, connection=None, pattern=None, local_paths=None)[source]¶ Download a list of files, either by providing the list explicitly, providing directories that contain files to download, or both.
- Args:
- files_to_download: list
- A list of full remote paths (can be relative) to files to download
- remote: str or list
- A path to a remote directory or a list of paths
- connection: obj
- An SFTP connection object
- pattern: str
- A regex pattern with which to select file names. Defaults to None, in which case all files will be selected.
- local_paths: list
- A list of paths to which to save the selected files. Defaults to None. If it is not the same length as the files to be fetched, temporary files are used instead.
- Returns:
- list
- Local paths where the files are saved.
-
get_table
(remote_path, connection=None)[source]¶ Download a csv from the server and convert into a Parsons table.
The file may be compressed with gzip, or zip, but may not contain multiple files in the archive.
- Args:
- remote_path: str
- The remote path of the file to download
- connection: obj
- An SFTP connection object
- Returns:
- Parsons Table
- See Parsons Table for output options.
-
put_file
(local_path, remote_path, connection=None)[source]¶ Put a file on the SFTP server
- Args:
- local_path: str
- The local path of the source file
- remote_path: str
- The remote path of the new file
- connection: obj
- An SFTP connection object
-
remove_file
(remote_path, connection=None)[source]¶ Delete a file on the SFTP server
- Args:
- remote_path: str
- The remote path of the file
- connection: obj
- An SFTP connection object
-
get_file_size
(remote_path, connection=None)[source]¶ Get the size of a file in MB on the SFTP server. The file is not downloaded locally.
- Args:
- remote_path: str
- The remote path of the file
- connection: obj
- An SFTP connection object
- Returns:
- int
- The file size in MB.
-
list_subdirectories
(remote_path, connection=None, pattern=None)[source]¶ List the subdirectories of a directory on the remote server.
- Args:
- remote_path: str
- The remote directory whose subdirectories will be listed
- connection: obj
- An SFTP connection object
- pattern: str
- A regex pattern with which to select full directory paths. Defaults to None, in which case all subdirectories will be selected.
- Returns:
- list
- The subdirectories in remote_path.
-
list_files
(remote_path, connection=None, pattern=None)[source]¶ List the files in a directory on the remote server.
- Args:
- remote_path: str
- The remote directory whose files will be listed
- connection: obj
- An SFTP connection object
- pattern: str
- A regex pattern with which to select file names. Defaults to None, in which case all files will be selected.
- Returns:
- list
- The files in remote_path.
-
walk_tree
(remote_path, connection=None, download=False, dir_pattern=None, file_pattern=None, max_depth=2)[source]¶ Recursively walks a directory, fetching all subdirectories and files (as long as they match dir_pattern and file_pattern, respectively) and the maximum directory depth hasn’t been exceeded. Optionally downloads discovered files.
- Args:
- remote_path: str
- The top level directory to walk
- connection: obj
- An SFTP connection object
- download: bool
- Whether to download discovered files
- dir_pattern: str
- A regex pattern with which to select directories. Defaults to None, in which case all directories will be selected.
- file_pattern: str
- A regex pattern with which to select files. Defaults to None, in which case all files will be selected.
- max_depth: int
- A limit on how many directories deep to traverse. The default, 2, will search the contents of remote_path and its subdirectories.
- Returns:
- tuple
- A list of directories touched and a list of files. If the files were downloaded the file list will consist of local paths, if not, remote paths.