How to Build a Python Script to Transfer Documents from OneDrive to S3

Cloud storage services like OneDrive and S3 (Simple Storage Service) are integral to many business workflows. Automating the transfer of documents from OneDrive to S3 not only simplifies workflows but also enables seamless integration across platforms. In this blog, we’ll walk through building a Python script to automate this process, including fetching metadata like document permissions and creator details.

Prerequisites

Before diving into the code, ensure you have:

1. Access to OneDrive API: Register your application in Microsoft Azure and get the client ID, tenant ID, and client secret.
2. Access to AWS S3: Set up an S3 bucket and configure an IAM role or user with sufficient permissions (e.g., s3:PutObject).
3. Python Environment: Install necessary Python libraries such as boto3 (for AWS S3), msal (for OneDrive API), and requests.

High-Level Steps

1. Authenticate with OneDrive.
2. Retrieve documents and metadata from OneDrive.
3. Upload the documents to S3 with metadata.
4. Log metadata for future use.

Step 1: Authenticate with OneDrive

Use the Microsoft Authentication Library (MSAL) to authenticate with OneDrive.

from msal import ConfidentialClientApplication

#…