Skip to main content

module aixplain.utils.file_utils

Copyright 2022 The aiXplain SDK authors

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

function save_file​

save_file(download_url: str, download_file_path: Optional[Any] = None) → Any

Download and save file from given URL

Args:

  • download_url (Text): URL of file to download
  • download_file_path (Any, optional): File path to save downloaded file. If None then generates a folder 'aiXplain' in current working directory. Defaults to None.

Returns:

  • Text: Path where file was downloaded

function download_data​

download_data(url_link, local_filename=None)

function upload_data​

upload_data(
    file_name: Union[str, Path],
    tags: Optional[List[str]] = None,
    license: Optional[License] = None,
    is_temp: bool = True,
    content_type: str = 'text/csv',
    content_encoding: Optional[str] = None,
    nattempts: int = 2
)

Upload files to S3 with pre-signed URLs

Args:

  • file_name (Union[Text, Path]): local path of file to be uploaded
  • tags (List[Text], optional): tags of the file
  • license (License, optional): the license for the file
  • is_temp (bool): specify if the file that will be upload is a temporary file
  • content_type (Text, optional): Type of content. Defaults to "text/csv".
  • content_encoding (Text, optional): Content encoding. Defaults to None.
  • nattempts (int, optional): Number of attempts for diminish the risk of exceptions. Defaults to 2.

Reference:

  • https: //python.plainenglish.io/upload-files-to-aws-s3-using-pre-signed-urls-in-python-d3c2fcab1b41

Returns:

  • URL: s3 path

function s3_to_csv​

s3_to_csv(s3_url: str, aws_credentials: Dict) → str

Convert s3 url to a csv file and download the file in download_path

Args:

  • s3_url (Text): s3 url

Returns:

  • Path: path to csv file