Gerald Hinojoza linkedin.com/in/geraldhinojoza/
Env: dev Region: centralus

Multi-Cloud Banking Data Platform

Secure AWS to Azure Data Pipeline via Workload Identity Federation (OIDC)

Sector: Banking / Financial Services

Architecture Style: Event-Driven Multi-Cloud (AWS + Azure)

Architecture Diagram (Spatial)

AWS Region Azure Region (centralus) S3 Bucket (Landing Zone) IAM Role Azure Data Factory (Managed Identity) Blob Storage (Raw Zone) Event Grid Function App Azure Databricks Cosmos DB (Serving Layer) Power BI Azure Monitor / Log Analytics Data Copy OIDC Trust Trigger ETL (API) Spark Write DirectQuery

Data Flow Sequence

Step 1 • AWS S3

File Ingestion

Bank transaction logs (CSV/JSON) are pushed to the AWS S3 landing bucket from external mainframe systems.

Step 2 • IAM & OIDC

Secretless Auth

Azure Data Factory assumes an AWS IAM role via Workload Identity Federation (OIDC). No long-lived access keys are stored.

Step 3 • Azure Data Factory

Cross-Cloud Copy

ADF Pipeline copies the delta files from AWS S3 into Azure Blob Storage (Raw Zone).

Step 4 • Event Grid & Functions

Event-Driven Trigger

Blob creation fires an Event Grid event to an Azure Function, which performs lightweight schema validation and triggers Databricks.

Step 5 • Azure Databricks

ETL & Fraud Detection

Databricks mounts Blob, cleanses data, runs ML-based fraud detection models, and structures the output.

Step 6 • Cosmos DB

Serving Layer

Cleaned, scored transactions are upserted into Azure Cosmos DB (NoSQL API) for high-concurrency, low-latency querying.

Step 7 • Power BI

Visualization

Power BI connects to Cosmos DB to provide real-time dashboards for the bank's fraud analytics team.

Step 8 • Azure Monitor

Governance & Logging

All component logs, pipeline metrics, and security events are centralized in Log Analytics.

Infrastructure as Code (Terraform)

# AWS Provider & S3 Landing Zone
resource "aws_s3_bucket" "banking_landing" {
  bucket = "gh-bank-landing-dev-centralus"
}

# AWS IAM Role for OIDC (Azure AD / Entra ID Trust)
resource "aws_iam_role" "azure_adf_role" {
  name = "AzureADFAccessRole"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRoleWithWebIdentity"
      Effect = "Allow"
      Principal = {
        Federated = "arn:aws:iam::123456789012:oidc-provider/login.microsoftonline.com/TENANT_ID/v2.0"
      }
      Condition = {
        StringEquals = {
          "login.microsoftonline.com/TENANT_ID/v2.0:sub" = "api://AzureADFAppID"
        }
      }
    }]
  })
}

# Azure Resource Group & Storage
resource "azurerm_resource_group" "rg" {
  name     = "rg-banking-poc-dev-centralus"
  location = "Central US"
}

resource "azurerm_storage_account" "raw" {
  name                     = "stbankrawdevcentralus"
  resource_group_name      = azurerm_resource_group.rg.name
  location                 = azurerm_resource_group.rg.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
  is_hns_enabled           = true
}

# Azure Data Factory & Managed Identity
resource "azurerm_user_assigned_identity" "adf_mi" {
  name                = "mi-adf-banking-dev"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
}

resource "azurerm_federated_identity_credential" "aws_trust" {
  name                = "fic-aws-trust"
  resource_group_name = azurerm_resource_group.rg.name
  audience            = ["api://AzureADFAppID"]
  issuer              = "https://login.microsoftonline.com/TENANT_ID/v2.0"
  parent_id           = azurerm_user_assigned_identity.adf_mi.id
  subject             = "system:serviceaccount:default:adf-sa"
}

resource "azurerm_cosmosdb_account" "cosmos" {
  name                = "cosmos-banking-dev-centralus"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  offer_type          = "Standard"
  kind                = "GlobalDocumentDB"

  capabilities {
    name = "EnableServerless"
  }

  consistency_policy {
    consistency_level = "Session"
  }
  
  geo_location {
    location          = azurerm_resource_group.rg.location
    failover_priority = 0
  }
}

Cost Estimation (Monthly)

Environment: dev | Region: centralus | Workload: ~10GB Data/mo, 100k requests.

Cloud Component SKU / Tier (Dev) Estimated Cost (USD)
AWS S3 Bucket Standard (10 GB) + API calls $ 0.50
AWS IAM Global / Free $ 0.00
Azure Data Factory Data Pipeline (DIU) - Scheduled runs $ 15.00
Azure Blob Storage (ADLS Gen2) Standard LRS (10 GB) $ 0.40
Azure Event Grid & Functions Consumption Tier (Free grant applies) $ 0.50
Azure Databricks Compute (Dev clusters, auto-terminate) $ 60.00
Azure Cosmos DB Serverless (Pay per RU) $ 5.00
Azure Monitor / Log Analytics Pay-As-You-Go (Data Ingestion ~5GB) $ 14.00
Total Estimated Monthly Cost (Dev Environment) $ 95.40

PoC Validation Checklist

1. Security & Authentication (OIDC)

2. Data Ingestion & Connectivity

3. Transformation & Serving

1 / 6