Automatically download Microsoft invoice PDFs into the hub data lake

[arthursilvany]

🔦 Context

FinOps hubs give us excellent cost and usage data through Cost Management exports, but the invoice documents themselves are not part of the data lake. Today, teams that need the actual invoice PDFs — for month-end close, audit evidence, internal chargeback packages, or reconciliation against the FOCUS dataset — download them manually from the Azure portal, one billing profile at a time.

For organizations with several billing profiles, this is a recurring monthly chore that is easy to forget and hard to audit. It also means the invoice and the cost data that explains it live in two different places.

💡 Proposal

Add an optional hub app that downloads Microsoft invoice PDFs from the Billing API into the existing ingestion container on a monthly schedule.

Proposed shape:

• New app under a Microsoft.Billing/Invoices namespace, following the existing hub app pattern (app.bicep + metadata.bicep, registered through fx/hub-app.bicep)
• Opt-in via a new step in the deployment wizard, disabled by default — existing hubs see no change unless they explicitly enable it
• Billing accounts configured in settings.json; when left blank, derived from the billing account scopes the hub already monitors
• Configurable day of month for the schedule (invoices are generally available within the first few days of the month)
• Files land in a predictable hierarchy for downstream reporting:

ingestion/
└── invoices/
    └── {YYYY-MM}/
        └── {billingProfileId}/
            └── {purchaseOrderNumber|no-po}/
                └── {invoiceNumber}.pdf

No new Azure resources are created — the app only adds artifacts to the Data Factory the hub already deploys. Estimated cost is roughly US$1/month in activity runs for a few dozen invoices.

🧩 Known constraints

• MCA/MPA only. The Microsoft.Billing invoice APIs don't cover legacy Enterprise Agreement. The pipeline should degrade gracefully (empty list, no failure) rather than error out.
• Billing Reader can't be granted by the template. The role assignment scope is /providers/Microsoft.Billing/billingAccounts/{id}, which is tenant-level and outside any subscription, so a resource-group–scoped deployment can't create it. This needs to be a documented post-deployment step, ideally backed by a helper cmdlet.
• Download URLs are short-lived SAS links (roughly one hour), so the download loop needs bounded parallelism.
• The activity that requests the download URL must use secureOutput since the response body contains a SAS token.

🌱 Possible follow-ups (out of scope for the initial change)

• A Power BI page reconciling invoices against FOCUS cost data by invoiceId
• Ingesting invoice Transactions (line items) via the API alongside the PDFs
• A control table tracking which invoices have already been downloaded

🙋 Offer to contribute

I have a working implementation of this and would be happy to open a PR if the team is interested in the direction. Happy to adjust the namespace, wizard placement, or configuration surface based on your preferences before I do — I'd rather align early than send a large PR that needs reshaping.

[microsoft-github-policy-service]

Thank you for submitting this issue! We try to respond within 3 business days. In the meantime:

• 📖 Check the troubleshooting guide for common issues
• 💬 Ask questions in Discussions
• 📅 Join our biweekly office hours for live help from the team