In this tutorial, we’ll walk through the steps to ingest incoming Office 365 emails to AWS S3 using AWS Lambda and CloudWatch alarms. This setup will help you archive your emails securely and access them as needed. Let’s get started.

Prerequisites

  1. AWS Account: Ensure you have an AWS account.
  2. Office 365 Account: An Office 365 account to access incoming emails.
  3. AWS CLI: Install and configure the AWS CLI.
  4. IAM Role: Create an IAM role with the necessary permissions for Lambda and S3.
  5. Azure AD App Registration: Register an application in Azure AD to access the Office 365 emails.

Step 1: Setting Up Azure AD for Office 365 API

  1. Register an Application:
    • Go to the Azure Portal, navigate to “Azure Active Directory” > “App registrations” > “New registration”.
    • Provide a name, and set the supported account types to “Accounts in this organizational directory only”.
    • Note the “Application (client) ID” and “Directory (tenant) ID”.
  2. Configure API Permissions:
    • Go to “API permissions” > “Add a permission” > “Microsoft Graph”.
    • Select “Delegated permissions” and add Mail.Read and Mail.ReadWrite.
    • Grant admin consent for the permissions.
  3. Generate a Client Secret:
    • Go to “Certificates & secrets” > “New client secret”.
    • Note the value of the client secret.

Step 2: Setting Up AWS S3

  1. Create S3 Bucket: Go to the AWS S3 console and create a new bucket to store your emails.
  2. Bucket Policy: Configure the bucket policy to allow the Lambda function to write to it.

Step 3: Configuring AWS Lambda

  1. Create Lambda Function:
    • Go to the AWS Lambda console and create a new function.
    • Choose the Python runtime.
    • Assign the IAM role created earlier.
  2. Upload Office 365 API Credentials:
    • Upload the client ID, tenant ID, and client secret to your Lambda function as environment variables or securely via AWS Secrets Manager.
  3. Lambda Code:
    • Use the following Python code as a starting point for your Lambda function:

import base64

import boto3

import requests

from requests.auth import HTTPBasicAuth

import os

s3 = boto3.client(‘s3’)

BUCKET_NAME = ‘your-s3-bucket-name’

CLIENT_ID = os.environ[‘CLIENT_ID’]

CLIENT_SECRET = os.environ[‘CLIENT_SECRET’]

TENANT_ID = os.environ[‘TENANT_ID’]

def get_access_token():

    url = f’https://login.microsoftonline.com/{TENANT_ID}/oauth2/v2.0/token’

    payload = {

        ‘client_id’: CLIENT_ID,

        ‘client_secret’: CLIENT_SECRET,

        ‘scope’: ‘https://graph.microsoft.com/.default’,

        ‘grant_type’: ‘client_credentials’

    }

    response = requests.post(url, data=payload)

    response.raise_for_status()

    return response.json()[‘access_token’]

def get_unread_emails(token):

    headers = {

        ‘Authorization’: f’Bearer {token}’,

        ‘Accept’: ‘application/json’

    }

    url = ‘https://graph.microsoft.com/v1.0/me/mailFolders/Inbox/messages?$filter=isRead eq false’

    response = requests.get(url, headers=headers)

    response.raise_for_status()

    return response.json()[‘value’]

def save_email_to_s3(email_data):

    email_id = email_data[‘id’]

    content = email_data[‘body’][‘content’]

    s3.put_object(Bucket=BUCKET_NAME, Key=f’emails/{email_id}.html’, Body=content.encode(‘utf-8’))

def lambda_handler(event, context):

    token = get_access_token()

    emails = get_unread_emails(token)

    for email in emails:

        save_email_to_s3(email)

        # Mark the email as read after processing

        email_id = email[‘id’]

        url = f’https://graph.microsoft.com/v1.0/me/messages/{email_id}’

        headers = {

            ‘Authorization’: f’Bearer {token}’,

            ‘Content-Type’: ‘application/json’

        }

        payload = {‘isRead’: True}

        requests.patch(url, headers=headers, json=payload)

Step 4: Setting Up CloudWatch Alarm

  1. Create a CloudWatch Event Rule:
    • Go to the CloudWatch console and create a new event rule.
    • Configure the rule to trigger the Lambda function at a regular interval (e.g., every 5 minutes).
  2. Add Target:
    • Add the Lambda function as the target for the event rule.

Step 5: Testing and Deployment

  1. Deploy Lambda Function: Deploy your Lambda function with the necessary dependencies.
  2. Test the Setup: Send a test email to your Office 365 account and verify that it appears in your S3 bucket.

Conclusion

By following these steps, you’ve successfully set up a system to ingest incoming Office 365 emails to AWS S3 using AWS Lambda and CloudWatch alarms. This solution ensures that your emails are securely archived and accessible when needed.