diff --git a/README.md b/README.md index 090b886..8bc72cf 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,11 @@ -This repo details how to seamlessly set up an Azure function as a good alternative to the MongoDB Atlas Data APIs. You can deploy the Azure Function App using few clicks and use the Azure function URL and Function App key instead of the base url and api key of Atlas Data APIs. It details the Azure function based replacement solution and alleviates the burden off developers to build the function code and manually deploy it. +This repo explains how to set up an Azure Function as an alternative to the MongoDB Atlas Data API. You can deploy the Azure Function App in a few clicks and use the Azure Function URL and Function App key instead of the Atlas Data API base URL and API key. It describes the Azure Function-based replacement solution and alleviates the burden on developers to build and deploy the function code manually. + ## Architecture ![](https://github.com/mongodb-partners/MongoDB_DataAPI_Azure/blob/main/images/DataAPIReplace.drawiov1.png) -The architecture diagram above depicts that when an API request is made to the Azure Function url, authorised by the API key, the corresponding MQL will be executed as specified by the “operation” in the url. An example of an Azure function URL being invoked would be : “https://.azurewebsites.net/api/mdb_dataapi/action/{operation}”. The url contains the “operation” which indicates which API is invoked (findOne, find, insertOne, insertMany, deleteOne, deleteMany, updateOne, updateMany, aggregate) and the corresponding python SDK code is executed using the parameters passed in the body of the request. The results of the MQL query (operation) are then passed back to the application. +The architecture diagram above shows that when an API request is made to the Azure Function URL, authorized by the API key, the corresponding MQL is executed based on the `operation` in the URL. An example Azure Function URL is: `https://.azurewebsites.net/api/mdb_dataapi/action/{operation}`. The URL contains the `operation`, which indicates which API is invoked (`findOne`, `find`, `insertOne`, `insertMany`, `deleteOne`, `deleteMany`, `updateOne`, `updateMany`, or `aggregate`), and the corresponding Python SDK code is executed using the parameters passed in the request body. The results of the MQL operation are then returned to the application. + ## Prerequisites @@ -19,17 +21,20 @@ Register for a new Atlas Account [here](https://www.mongodb.com/docs/atlas/tutor ### **Option 1: Set Up Azure function Using GitHub actions** ### - a.Fork the [MongoDB repo](https://github.com/mongodb-partners/MongoDB_DataAPI_Azure). Note the new **forked repo url**. If GitHub actions is NOT enabled by default, enable them by going to the **Settings -> Actions -> General** in your forked repo and select one of Allow actions/ resusable workflows options. + a. Fork the [MongoDB repo](https://github.com/mongodb-partners/MongoDB_DataAPI_Azure). Note the new **forked repo URL**. If GitHub Actions is not enabled by default, enable it by going to **Settings -> Actions -> General** in your forked repo and selecting one of the options that allows actions or reusable workflows. + + + b. Click the **Deploy to Azure** button below to create the Azure Function in your tenant. - b.Click the below **Deploy to Azure** button to have the Azure function created in your tenant. [![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fmongodb-partners%2FMongoDB_DataAPI_Azure%2Frefs%2Fheads%2Fmain%2FARM_template.json) c. Select or Create your Resource group which will contain the Azure function and its associated components (App Service Plan, Storage Account and App Insights). You can keep the function name and SKU as the defaults or change if you like to follow some specific standards. **We recommend that you add your Cluster name to the function app name so that its unique and easy to identify.** - Give the MongoDB connection url for the Cluster against which this Azure function will run. This connection string will be saved as an Environmnet variable. - Give your forked repo url as GitHub repo. Select **Create** and it will create the Azure function with the associated resources. + Give the MongoDB connection URL for the cluster against which this Azure Function will run. This connection string will be saved as an environment variable. + Give your forked repo URL as the GitHub repo. Select **Create** and it will create the Azure Function with the associated resources. + **Note that at this stage the function app is created, env variables are populated but the actual function is not yet deployed to the function app.** d. To have GitHub actions run from your repo and deploy the function, get the publishing profile from your created Azure function. @@ -41,40 +46,48 @@ Register for a new Atlas Account [here](https://www.mongodb.com/docs/atlas/tutor e. Go to your GitHub repo -> Settings -> Secrets and variables -> Actions Click **New Respository secret** and copy the entire value in your publishing profile to a new secret named **"AZUREAPPSERVICE_PUBLISHPROFILE"** - f. Make a minor change in README and **Commit Changes** to invoke GitHub actions which would deploy the python code to the Azure function into your function app. - **Now you should see the function available in the Functon App and the code in function_app.py deployed.** + f. Make a minor change in `README.md` and **Commit Changes** to trigger GitHub Actions, which will deploy the Python code to your Azure Function App. + **Now you should see the function available in the Function App and the code in `function_app.py` deployed.** + - g. GitHub actions tab in GitHub repo will show the steps in the deployment (including the installation of dependencies) and the result of each step. + g. The GitHub Actions tab in your GitHub repo will show the deployment steps, including dependency installation, and the result of each step. + ### **Option 2: Set Up Azure function Using ZipDeploy** ### - a. Click the below **Deploy to Azure** button to have the Azure function created in your tenant. + a. Click the **Deploy to Azure** button below to create the Azure Function in your tenant. + [![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fmongodb-partners%2FMongoDB_DataAPI_Azure%2Frefs%2Fheads%2Fmain%2FARM_template_zipdeploy.json) - b. Select or Create your Resource group which will contain the Azure function and its associated components (App Service Plan, Storage Account and App Insights). You can keep the function name and SKU as the defaults or change if you like to follow some specific standards. We recommned that you add your Cluster name to the function app name so that its unique and easy to identify. + b. Select or create your resource group, which will contain the Azure Function and its associated components (App Service Plan, Storage Account, and Application Insights). You can keep the function name and SKU as the defaults or change them if you want to follow specific standards. We recommend that you add your cluster name to the Function App name so it is unique and easy to identify. + Please **DO NOT** change the **Package Url** which points to the deployable zip file in this repo. - Select **Create** and it will create the Azure function app, deploy the azure function along with the associated resources. + Select **Create** and Azure will create the Function App and deploy the Azure Function along with the associated resources. + ## How to get credentials **Get the BaseUrl and API Key** -a. From the Function App, select your function and click **Get function URL** . Copy the function url from the beginning till "/action/" as shown in screenshot below. Replace {operation} with one of these depending on which API needs to be invoked. The valid operations are : findOne, find, insertOne, insertMany, deleteOne, deleteMany, updateOne, updateMany, aggregate -This is the **Base Url** you will use to invoke any of the MongoDB CRUD/ Aggregate APIs. +a. From the Function App, select your function and click **Get function URL**. Copy the function URL from the beginning through `/action/` as shown in the screenshot below. Replace `{operation}` with one of the supported API names depending on which API needs to be invoked. The valid operations are: `findOne`, `find`, `insertOne`, `insertMany`, `deleteOne`, `deleteMany`, `updateOne`, `updateMany`, and `aggregate`. +This is the **Base URL** you will use to invoke the MongoDB CRUD and aggregate APIs. + ![](https://github.com/mongodb-partners/MongoDB_DataAPI_Azure/blob/1d2b33a4e8bc7c2cbc325dac9840ded28ebdb1f7/images/Get%20Function%20url.png) -b. Go to your Function App -> Under Functions -> App keys , Grab either the *_master* or *default* API key for your Azure function - This is the **API Key** you will use along with **Base Url** to create a MongoDB connection to invoke any of the MongoDB CRUD/ Aggregate APIs. +b. Go to your Function App -> Under Functions -> App keys, and copy either the *_master* or *default* API key for your Azure Function. + This is the **API key** you will use along with the **Base URL** to invoke the MongoDB CRUD and aggregate APIs. + ## Invoking the APIs using Azure function -Change your applications to use an url format like : "https://.azurewebsites.net/api/mdb_dataapi/action/{operation}" to invoke the Data APIs. Not the operation will have unique values for each of the Data API operations which are - findOne, find, insertOne, insertMany, deleteOne, deleteMany, updateOne, updateMany, aggregate. For example use "https://.azurewebsites.net/api/mdb_dataapi/action/findOne" to query the database and retrieve only one record. +Change your applications to use a URL format like `https://.azurewebsites.net/api/mdb_dataapi/action/{operation}` to invoke the Data APIs. Note that `operation` must be one of the supported values: `findOne`, `find`, `insertOne`, `insertMany`, `deleteOne`, `deleteMany`, `updateOne`, `updateMany`, or `aggregate`. For example, use `https://.azurewebsites.net/api/mdb_dataapi/action/findOne` to query the database and retrieve one record. + +Also note that in the authorization headers you should add `x-functions-key`, and its value should be the API key of the Azure Function. The request body should include inputs such as `database` and `collection`, along with any operation-specific parameters. The `dataSource` field may be sent for compatibility with Atlas Data API clients, but this function uses the configured environment connection string to determine the MongoDB cluster. -Also note that in the authorisation add key name as "x-functions-key" and its value should be the API key of the Azure function. This should be part of header too. Rest of the inputs like "dataSource" for clustername, "database" for database name and "collection" for collection name should be passed in the request body along with other optional parameters. See below curl as an example: @@ -90,10 +103,39 @@ See below curl as an example: }' ``` Refer to the [swagger file](https://github.com/mongodb-partners/MongoDB_DataAPI_Azure/blob/main/MongoDB_clean_swagger.json) for the structure of each of the APIs. + +## ObjectId request and response behavior + +This function now handles MongoDB `ObjectId` values explicitly so that queries and responses behave more like a MongoDB-aware API. + +### Responses +- Any MongoDB `ObjectId` returned by MongoDB is serialized as a string in the JSON response. +- This applies recursively, including nested documents and arrays, not just top-level `_id` fields. + +### Requests +- When filtering by `_id`, you may send either: + - a plain string value, for example: + - `{"_id": "507f1f77bcf86cd799439011"}` + - extended JSON for an actual ObjectId, for example: + - `{"_id": {"$oid": "507f1f77bcf86cd799439011"}}` +- Extended JSON `{"$oid": "..."}` is also supported in nested request payloads such as: + - filters + - update documents + - insert documents + - aggregation pipelines + +### Important note +- Plain string `_id` values are preserved as strings. +- The function does **not** automatically convert plain `_id` strings into `ObjectId`, because MongoDB collections may legitimately use string `_id` values. +- Only explicit `{"$oid": "..."}` wrappers are converted to MongoDB `ObjectId` values. + + ## Known issues and limitations + Please follow this [link](https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale) for the known limitations with the Azure functions like time outs and other service limits for each resource plans. ## Common errors and remedies -Typical API response codes apply here also. Any 4XX errors indicate issue with the request from the client. Make sure that the dataSource, database, collection are provided in a valid JSON format. Refer to this [Postman Collection](https://www.postman.com/grey-desert-5714/workspace/mongodb-dataapis-azurefcn/collection/5631262-a038ba24-f185-4671-acf2-530b3a3ddb55?action=share&creator=5631262) for examples. For 5XX errors, make sure the Azure function is up and running and check its trace to further investigate. +Typical API response codes apply here as well. Any 4XX errors indicate an issue with the client request. Make sure that the request body is valid JSON and that required fields such as `database` and `collection` are provided. Refer to this [Postman Collection](https://www.postman.com/grey-desert-5714/workspace/mongodb-dataapis-azurefcn/collection/5631262-a038ba24-f185-4671-acf2-530b3a3ddb55?action=share&creator=5631262) for examples. For 5XX errors, make sure the Azure Function is running and check its logs for more details. + diff --git a/function_app.py b/function_app.py index 6ebeffe..ad69213 100644 --- a/function_app.py +++ b/function_app.py @@ -1,143 +1,300 @@ -import azure.functions as func -import logging +""" +Changelog: +- 2026-08-18: Stopped auto-converting plain `_id` strings to ObjectId so string-based `_id` values remain supported. +- 2026-08-18: Replaced aggregate-based find logic with native MongoDB find(), sort(), skip(), and limit(). +- 2026-08-18: Added recursive _id filter normalization for nested operators such as $or, $and, $in, and $eq. +- 2026-08-18: Split bad-request and internal-server-error responses for clearer API behavior. +- 2026-08-18: Fixed ObjectId handling so responses serialize nested MongoDB ObjectIds reliably. +- 2026-08-18: Added request-side ObjectId normalization for filters, updates, inserted documents, and pipelines. +- 2026-08-18: Corrected updateOne/deleteOne route comparisons that previously prevented single-document operations from running. +- 2026-08-18: Improved inline documentation and helper function names for maintainability. +""" + + + import json -import traceback +import logging import os -from pymongo import MongoClient -from bson import ObjectId +import traceback from datetime import datetime +import azure.functions as func +from bson import ObjectId +from pymongo import MongoClient + app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION) +# Controls whether ObjectId values are converted to strings before JSON serialization. +# This should remain enabled for HTTP responses because ObjectId is not JSON serializable. +ENABLE_OBJECTID_CONVERSION = os.getenv("ENABLE_OBJECTID_CONVERSION", "true").lower() == "true" + + +class MongoJSONEncoder(json.JSONEncoder): + """JSON encoder that serializes MongoDB-specific and datetime values safely.""" + + def default(self, value): + if isinstance(value, datetime): + return value.isoformat() + if ENABLE_OBJECTID_CONVERSION and isinstance(value, ObjectId): + return str(value) + return super().default(value) + + +def serialize_mongo_types(value): + """ + Recursively convert MongoDB-specific Python types into JSON-safe values. + + Why this exists: + - MongoDB returns ObjectId instances, including nested ObjectIds inside arrays or subdocuments. + - json.dumps() cannot serialize ObjectId directly. + - Converting only top-level `_id` fields is not enough because ObjectIds may appear anywhere. + """ + if isinstance(value, list): + return [serialize_mongo_types(item) for item in value] + + if isinstance(value, dict): + return {key: serialize_mongo_types(item) for key, item in value.items()} + + if ENABLE_OBJECTID_CONVERSION and isinstance(value, ObjectId): + return str(value) + + return value + + +def normalize_objectid_inputs(value): + """ + Recursively convert extended-JSON ObjectId payloads into bson.ObjectId. + + Supported request formats: + - {"_id": "507f1f77bcf86cd799439011"} + - {"_id": {"$oid": "507f1f77bcf86cd799439011"}} + - nested structures inside filters, updates, inserted documents, and pipelines + + Plain strings are preserved unless they are used in MongoDB ObjectId-specific wrappers. + This avoids incorrectly converting ordinary string fields that merely look like ObjectIds. + """ + if isinstance(value, list): + return [normalize_objectid_inputs(item) for item in value] + + if isinstance(value, dict): + if set(value.keys()) == {"$oid"} and ObjectId.is_valid(value["$oid"]): + return ObjectId(value["$oid"]) + return {key: normalize_objectid_inputs(item) for key, item in value.items()} + + return value + + +def convert_id_value_for_filter(value): + """ + Normalize `_id` filter values without guessing whether plain strings should be ObjectId. + + Only explicit extended JSON ObjectId wrappers such as `{ "$oid": "..." }` + are converted. Plain strings are preserved so collections that store string + `_id` values continue to work correctly. + """ + value = normalize_objectid_inputs(value) + + if isinstance(value, list): + return [convert_id_value_for_filter(item) for item in value] + + if isinstance(value, dict): + return {key: convert_id_value_for_filter(item) for key, item in value.items()} + + return value + + + + +def normalize_id_filter(filter_value): + """ + Recursively normalize `_id` filters across nested MongoDB operators. + + This supports filters such as: + - {"_id": "507f1f77bcf86cd799439011"} + - {"_id": {"$in": ["507f1f77bcf86cd799439011"]}} + - {"_id": {"$in": [{"$oid": "507f1f77bcf86cd799439011"}]}} + - {"$or": [{"_id": "507f1f77bcf86cd799439011"}, {"status": "A"}]} + + Plain string `_id` values are preserved. Only explicit `{"$oid": "..."}` + values are converted to ObjectId. + """ + + filter_value = normalize_objectid_inputs(filter_value) + + if isinstance(filter_value, list): + return [normalize_id_filter(item) for item in filter_value] + + if not isinstance(filter_value, dict): + return filter_value + + normalized_filter = {} + for key, value in filter_value.items(): + if key == "_id": + normalized_filter[key] = convert_id_value_for_filter(value) + else: + normalized_filter[key] = normalize_id_filter(value) + + return normalized_filter + + + def connect_to_mongodb(): + """Create and return a MongoDB client using the configured Atlas connection string.""" conn_str = os.environ.get("MONGODBATLAS_CLUSTER_CONNECTIONSTRING") if not conn_str: raise Exception("MongoDB connection string not found in environment variables.") + try: - client = MongoClient(conn_str) - return client - except Exception as e: - logging.error(f"Error connecting to MongoDB: {e}") + return MongoClient(conn_str) + except Exception as exc: + logging.error(f"Error connecting to MongoDB: {exc}") raise def success_response(body): + """Return a JSON HTTP response with MongoDB-safe serialization applied.""" + safe_body = serialize_mongo_types(body) return func.HttpResponse( - json.dumps(body, cls=DateTimeEncoder), + json.dumps(safe_body, cls=MongoJSONEncoder), status_code=200, - mimetype="application/json" + mimetype="application/json", ) -def error_response(err): - error_message = str(err) + +def error_response(err, status_code=400): + """Return a standard JSON error response with the supplied HTTP status code.""" return func.HttpResponse( - error_message, - status_code=400, - mimetype="application/json" + json.dumps({"error": str(err)}), + status_code=status_code, + mimetype="application/json", ) -# Used to convert datetime object(s) to string -class DateTimeEncoder(json.JSONEncoder): - def default(self, o): - if isinstance(o, datetime): - return o.isoformat() - return super().default(o) - -@app.route(route="mdb_dataapi/action/{operation}",methods=['POST']) + + + + + +@app.route(route="mdb_dataapi/action/{operation}", methods=["POST"]) def mongodb_dataapi_replace(req: func.HttpRequest) -> func.HttpResponse: - logging.info('Python HTTP trigger function processed a request.') + """ + Azure Function entry point that mimics selected MongoDB Data API operations. + + Supported operations: + - findOne + - find + - insertOne + - insertMany + - updateOne + - updateMany + - deleteOne + - deleteMany + - aggregate + """ + logging.info("Python HTTP trigger function processed a request.") client = None try: payload = req.get_json() + if not isinstance(payload, dict): + return error_response("Request body must be a JSON object.", status_code=400) + client = connect_to_mongodb() - op = req.route_params.get('operation') - # logging.info(op) - db,coll = payload.get('database'),payload.get('collection') - # logging.info(db) - # logging.info(coll) - if op == "findOne": - filter_op = payload['filter'] if 'filter' in payload else {} - projection = payload['projection'] if 'projection' in payload else {} - result = {"document": client[db][coll].find_one(filter_op, projection)} - # print("*************") - # print(result) - # print("*************") - if result['document'] is not None: - if '_id' in result['document'] and isinstance(result['document']['_id'], ObjectId): - result['document']['_id'] = str(result['document']['_id']) - elif op == "find": - agg_query = [] + op = req.route_params.get("operation") + db = payload.get("database") + coll = payload.get("collection") - if 'filter' in payload and payload['filter'] != {}: - agg_query.append({"$match": payload['filter']}) + if not db or not coll: + return error_response("Both 'database' and 'collection' are required.", status_code=400) - if "sort" in payload and payload['sort'] != {}: - agg_query.append({"$sort": payload['sort']}) + collection = client[db][coll] - if "skip" in payload: - agg_query.append({"$skip": payload['skip']}) + if op == "findOne": + filter_op = normalize_id_filter(payload.get("filter", {})) + projection = payload.get("projection", {}) + result = {"document": collection.find_one(filter_op, projection)} - if 'limit' in payload: - agg_query.append({"$limit": payload['limit']}) + elif op == "find": + filter_op = normalize_id_filter(payload.get("filter", {})) + projection = payload.get("projection", {}) + cursor = collection.find(filter_op, projection) - if "projection" in payload and payload['projection'] != {}: - agg_query.append({"$project": payload['projection']}) + if payload.get("sort"): + cursor = cursor.sort(list(payload["sort"].items())) + if "skip" in payload: + cursor = cursor.skip(payload["skip"]) + if "limit" in payload: + cursor = cursor.limit(payload["limit"]) - result = {"documents": list(client[db][coll].aggregate(agg_query))} - for obj in result['documents']: - if '_id' in obj and isinstance(obj['_id'], ObjectId): - obj['_id'] = str(obj['_id']) + result = {"documents": list(cursor)} elif op == "insertOne": - if "document" not in payload or payload['document'] == {}: - return error_response("Send a document to insert") - insert_op = client[db][coll].insert_one(payload['document']) - result = {"insertedId": str(insert_op.inserted_id)} + document = payload.get("document") + if not document: + return error_response("Send a document to insert", status_code=400) + + insert_op = collection.insert_one(normalize_objectid_inputs(document)) + result = {"insertedId": insert_op.inserted_id} elif op == "insertMany": - if "documents" not in payload or payload['documents'] == {}: - return error_response("Send a document to insert") - insert_op = client[db][coll].insert_many(payload['documents']) - result = {"insertedIds": [str(_id) for _id in insert_op.inserted_ids]} + documents = payload.get("documents") + if not documents: + return error_response("Send documents to insert", status_code=400) + + insert_op = collection.insert_many(normalize_objectid_inputs(documents)) + result = {"insertedIds": insert_op.inserted_ids} elif op in ["updateOne", "updateMany"]: - payload['upsert'] = payload['upsert'] if 'upsert' in payload else False - if "_id" in payload['filter']: - payload['filter']['_id'] = ObjectId(payload['filter']['_id']) - if op == "/updateOne": - update_op = client[db][coll].update_one(payload['filter'], payload['update'], upsert=payload['upsert']) + filter_op = normalize_id_filter(payload.get("filter", {})) + update_op_payload = normalize_objectid_inputs(payload.get("update", {})) + upsert = payload.get("upsert", False) + + if not update_op_payload: + return error_response("Send an update document", status_code=400) + + if op == "updateOne": + update_op = collection.update_one(filter_op, update_op_payload, upsert=upsert) else: - update_op = client[db][coll].update_many(payload['filter'], payload['update'], upsert=payload['upsert']) - result = {"matchedCount": update_op.matched_count, "modifiedCount": update_op.modified_count} + update_op = collection.update_many(filter_op, update_op_payload, upsert=upsert) + + result = { + "matchedCount": update_op.matched_count, + "modifiedCount": update_op.modified_count, + "upsertedId": update_op.upserted_id, + } elif op in ["deleteOne", "deleteMany"]: - payload['filter'] = payload['filter'] if 'filter' in payload else {} - if "_id" in payload['filter']: - payload['filter']['_id'] = ObjectId(payload['filter']['_id']) - if op == "/deleteOne": - result = {"deletedCount": client[db][coll].delete_one(payload['filter']).deleted_count} + filter_op = normalize_id_filter(payload.get("filter", {})) + + if op == "deleteOne": + delete_op = collection.delete_one(filter_op) else: - result = {"deletedCount": client[db][coll].delete_many(payload['filter']).deleted_count} + delete_op = collection.delete_many(filter_op) + + result = {"deletedCount": delete_op.deleted_count} elif op == "aggregate": - if "pipeline" not in payload or payload['pipeline'] == []: - return error_response("Send a pipeline") - docs = list(client[db][coll].aggregate(payload['pipeline'])) - for obj in docs: - if '_id' in obj and isinstance(obj['_id'], ObjectId): - obj['_id'] = str(obj['_id']) - result = {"documents": docs} + pipeline = payload.get("pipeline", []) + if not pipeline: + return error_response("Send a pipeline", status_code=400) + + result = {"documents": list(collection.aggregate(normalize_objectid_inputs(pipeline)))} else: - return error_response("Not a valid operation") + return error_response("Not a valid operation", status_code=400) return success_response(result) - except Exception as e: - print(traceback.format_exc()) - return error_response(e) + except ValueError as exc: + logging.warning(traceback.format_exc()) + return error_response(exc, status_code=400) + + except Exception: + logging.error(traceback.format_exc()) + return error_response("Internal server error", status_code=500) finally: if client: client.close() + + + diff --git a/mdbdataapireplacev1.zip b/mdbdataapireplacev1.zip index 031e9ca..ca166f5 100644 Binary files a/mdbdataapireplacev1.zip and b/mdbdataapireplacev1.zip differ diff --git a/mdbdataapireplacev1/function_app.py b/mdbdataapireplacev1/function_app.py index 6d9ee9b..3a84cdb 100644 --- a/mdbdataapireplacev1/function_app.py +++ b/mdbdataapireplacev1/function_app.py @@ -1,143 +1,299 @@ -import azure.functions as func -import logging +""" +Changelog: +- 2026-08-18: Stopped auto-converting plain `_id` strings to ObjectId so string-based `_id` values remain supported. +- 2026-08-18: Replaced aggregate-based find logic with native MongoDB find(), sort(), skip(), and limit(). +- 2026-08-18: Added recursive _id filter normalization for nested operators such as $or, $and, $in, and $eq. +- 2026-08-18: Split bad-request and internal-server-error responses for clearer API behavior. +- 2026-08-18: Fixed ObjectId handling so responses serialize nested MongoDB ObjectIds reliably. +- 2026-08-18: Added request-side ObjectId normalization for filters, updates, inserted documents, and pipelines. +- 2026-08-18: Corrected updateOne/deleteOne route comparisons that previously prevented single-document operations from running. +- 2026-08-18: Improved inline documentation and helper function names for maintainability. +""" + + + import json -import traceback +import logging import os -from pymongo import MongoClient -from bson import ObjectId +import traceback from datetime import datetime +import azure.functions as func +from bson import ObjectId +from pymongo import MongoClient + app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION) +# Controls whether ObjectId values are converted to strings before JSON serialization. +# This should remain enabled for HTTP responses because ObjectId is not JSON serializable. +ENABLE_OBJECTID_CONVERSION = os.getenv("ENABLE_OBJECTID_CONVERSION", "true").lower() == "true" + + +class MongoJSONEncoder(json.JSONEncoder): + """JSON encoder that serializes MongoDB-specific and datetime values safely.""" + + def default(self, value): + if isinstance(value, datetime): + return value.isoformat() + if ENABLE_OBJECTID_CONVERSION and isinstance(value, ObjectId): + return str(value) + return super().default(value) + + +def serialize_mongo_types(value): + """ + Recursively convert MongoDB-specific Python types into JSON-safe values. + + Why this exists: + - MongoDB returns ObjectId instances, including nested ObjectIds inside arrays or subdocuments. + - json.dumps() cannot serialize ObjectId directly. + - Converting only top-level `_id` fields is not enough because ObjectIds may appear anywhere. + """ + if isinstance(value, list): + return [serialize_mongo_types(item) for item in value] + + if isinstance(value, dict): + return {key: serialize_mongo_types(item) for key, item in value.items()} + + if ENABLE_OBJECTID_CONVERSION and isinstance(value, ObjectId): + return str(value) + + return value + + +def normalize_objectid_inputs(value): + """ + Recursively convert extended-JSON ObjectId payloads into bson.ObjectId. + + Supported request formats: + - {"_id": "507f1f77bcf86cd799439011"} + - {"_id": {"$oid": "507f1f77bcf86cd799439011"}} + - nested structures inside filters, updates, inserted documents, and pipelines + + Plain strings are preserved unless they are used in MongoDB ObjectId-specific wrappers. + This avoids incorrectly converting ordinary string fields that merely look like ObjectIds. + """ + if isinstance(value, list): + return [normalize_objectid_inputs(item) for item in value] + + if isinstance(value, dict): + if set(value.keys()) == {"$oid"} and ObjectId.is_valid(value["$oid"]): + return ObjectId(value["$oid"]) + return {key: normalize_objectid_inputs(item) for key, item in value.items()} + + return value + + +def convert_id_value_for_filter(value): + """ + Normalize `_id` filter values without guessing whether plain strings should be ObjectId. + + Only explicit extended JSON ObjectId wrappers such as `{ "$oid": "..." }` + are converted. Plain strings are preserved so collections that store string + `_id` values continue to work correctly. + """ + value = normalize_objectid_inputs(value) + + if isinstance(value, list): + return [convert_id_value_for_filter(item) for item in value] + + if isinstance(value, dict): + return {key: convert_id_value_for_filter(item) for key, item in value.items()} + + return value + + + + +def normalize_id_filter(filter_value): + """ + Recursively normalize `_id` filters across nested MongoDB operators. + + This supports filters such as: + - {"_id": "507f1f77bcf86cd799439011"} + - {"_id": {"$in": ["507f1f77bcf86cd799439011"]}} + - {"_id": {"$in": [{"$oid": "507f1f77bcf86cd799439011"}]}} + - {"$or": [{"_id": "507f1f77bcf86cd799439011"}, {"status": "A"}]} + + Plain string `_id` values are preserved. Only explicit `{"$oid": "..."}` + values are converted to ObjectId. + """ + + filter_value = normalize_objectid_inputs(filter_value) + + if isinstance(filter_value, list): + return [normalize_id_filter(item) for item in filter_value] + + if not isinstance(filter_value, dict): + return filter_value + + normalized_filter = {} + for key, value in filter_value.items(): + if key == "_id": + normalized_filter[key] = convert_id_value_for_filter(value) + else: + normalized_filter[key] = normalize_id_filter(value) + + return normalized_filter + + + def connect_to_mongodb(): + """Create and return a MongoDB client using the configured Atlas connection string.""" conn_str = os.environ.get("MONGODBATLAS_CLUSTER_CONNECTIONSTRING") if not conn_str: raise Exception("MongoDB connection string not found in environment variables.") + try: - client = MongoClient(conn_str) - return client - except Exception as e: - logging.error(f"Error connecting to MongoDB: {e}") + return MongoClient(conn_str) + except Exception as exc: + logging.error(f"Error connecting to MongoDB: {exc}") raise def success_response(body): + """Return a JSON HTTP response with MongoDB-safe serialization applied.""" + safe_body = serialize_mongo_types(body) return func.HttpResponse( - json.dumps(body, cls=DateTimeEncoder), + json.dumps(safe_body, cls=MongoJSONEncoder), status_code=200, - mimetype="application/json" + mimetype="application/json", ) -def error_response(err): - error_message = str(err) + +def error_response(err, status_code=400): + """Return a standard JSON error response with the supplied HTTP status code.""" return func.HttpResponse( - error_message, - status_code=400, - mimetype="application/json" + json.dumps({"error": str(err)}), + status_code=status_code, + mimetype="application/json", ) -# Used to convert datetime object(s) to string -class DateTimeEncoder(json.JSONEncoder): - def default(self, o): - if isinstance(o, datetime): - return o.isoformat() - return super().default(o) - -@app.route(route="mdb_dataapi/action/{operation}",methods=['POST']) + + + + + +@app.route(route="mdb_dataapi/action/{operation}", methods=["POST"]) def mongodb_dataapi_replace(req: func.HttpRequest) -> func.HttpResponse: - logging.info('Python HTTP trigger function processed a request.') + """ + Azure Function entry point that mimics selected MongoDB Data API operations. + + Supported operations: + - findOne + - find + - insertOne + - insertMany + - updateOne + - updateMany + - deleteOne + - deleteMany + - aggregate + """ + logging.info("Python HTTP trigger function processed a request.") client = None try: payload = req.get_json() + if not isinstance(payload, dict): + return error_response("Request body must be a JSON object.", status_code=400) + client = connect_to_mongodb() - op = req.route_params.get('operation') - # logging.info(op) - db,coll = payload.get('database'),payload.get('collection') - # logging.info(db) - # logging.info(coll) - if op == "findOne": - filter_op = payload['filter'] if 'filter' in payload else {} - projection = payload['projection'] if 'projection' in payload else {} - result = {"document": client[db][coll].find_one(filter_op, projection)} - # print("*************") - # print(result) - # print("*************") - if result['document'] is not None: - if isinstance(result['document']['_id'], ObjectId): - result['document']['_id'] = str(result['document']['_id']) - elif op == "find": - agg_query = [] + op = req.route_params.get("operation") + db = payload.get("database") + coll = payload.get("collection") - if 'filter' in payload and payload['filter'] != {}: - agg_query.append({"$match": payload['filter']}) + if not db or not coll: + return error_response("Both 'database' and 'collection' are required.", status_code=400) - if "sort" in payload and payload['sort'] != {}: - agg_query.append({"$sort": payload['sort']}) + collection = client[db][coll] - if "skip" in payload: - agg_query.append({"$skip": payload['skip']}) + if op == "findOne": + filter_op = normalize_id_filter(payload.get("filter", {})) + projection = payload.get("projection", {}) + result = {"document": collection.find_one(filter_op, projection)} - if 'limit' in payload: - agg_query.append({"$limit": payload['limit']}) + elif op == "find": + filter_op = normalize_id_filter(payload.get("filter", {})) + projection = payload.get("projection", {}) + cursor = collection.find(filter_op, projection) - if "projection" in payload and payload['projection'] != {}: - agg_query.append({"$project": payload['projection']}) + if payload.get("sort"): + cursor = cursor.sort(list(payload["sort"].items())) + if "skip" in payload: + cursor = cursor.skip(payload["skip"]) + if "limit" in payload: + cursor = cursor.limit(payload["limit"]) - result = {"documents": list(client[db][coll].aggregate(agg_query))} - for obj in result['documents']: - if isinstance(obj['_id'], ObjectId): - obj['_id'] = str(obj['_id']) + result = {"documents": list(cursor)} elif op == "insertOne": - if "document" not in payload or payload['document'] == {}: - return error_response("Send a document to insert") - insert_op = client[db][coll].insert_one(payload['document']) - result = {"insertedId": str(insert_op.inserted_id)} + document = payload.get("document") + if not document: + return error_response("Send a document to insert", status_code=400) + + insert_op = collection.insert_one(normalize_objectid_inputs(document)) + result = {"insertedId": insert_op.inserted_id} elif op == "insertMany": - if "documents" not in payload or payload['documents'] == {}: - return error_response("Send a document to insert") - insert_op = client[db][coll].insert_many(payload['documents']) - result = {"insertedIds": [str(_id) for _id in insert_op.inserted_ids]} + documents = payload.get("documents") + if not documents: + return error_response("Send documents to insert", status_code=400) + + insert_op = collection.insert_many(normalize_objectid_inputs(documents)) + result = {"insertedIds": insert_op.inserted_ids} elif op in ["updateOne", "updateMany"]: - payload['upsert'] = payload['upsert'] if 'upsert' in payload else False - if "_id" in payload['filter']: - payload['filter']['_id'] = ObjectId(payload['filter']['_id']) - if op == "/updateOne": - update_op = client[db][coll].update_one(payload['filter'], payload['update'], upsert=payload['upsert']) + filter_op = normalize_id_filter(payload.get("filter", {})) + update_op_payload = normalize_objectid_inputs(payload.get("update", {})) + upsert = payload.get("upsert", False) + + if not update_op_payload: + return error_response("Send an update document", status_code=400) + + if op == "updateOne": + update_op = collection.update_one(filter_op, update_op_payload, upsert=upsert) else: - update_op = client[db][coll].update_many(payload['filter'], payload['update'], upsert=payload['upsert']) - result = {"matchedCount": update_op.matched_count, "modifiedCount": update_op.modified_count} + update_op = collection.update_many(filter_op, update_op_payload, upsert=upsert) + + result = { + "matchedCount": update_op.matched_count, + "modifiedCount": update_op.modified_count, + "upsertedId": update_op.upserted_id, + } elif op in ["deleteOne", "deleteMany"]: - payload['filter'] = payload['filter'] if 'filter' in payload else {} - if "_id" in payload['filter']: - payload['filter']['_id'] = ObjectId(payload['filter']['_id']) - if op == "/deleteOne": - result = {"deletedCount": client[db][coll].delete_one(payload['filter']).deleted_count} + filter_op = normalize_id_filter(payload.get("filter", {})) + + if op == "deleteOne": + delete_op = collection.delete_one(filter_op) else: - result = {"deletedCount": client[db][coll].delete_many(payload['filter']).deleted_count} + delete_op = collection.delete_many(filter_op) + + result = {"deletedCount": delete_op.deleted_count} elif op == "aggregate": - if "pipeline" not in payload or payload['pipeline'] == []: - return error_response("Send a pipeline") - docs = list(client[db][coll].aggregate(payload['pipeline'])) - for obj in docs: - if isinstance(obj['_id'], ObjectId): - obj['_id'] = str(obj['_id']) - result = {"documents": docs} + pipeline = payload.get("pipeline", []) + if not pipeline: + return error_response("Send a pipeline", status_code=400) + + result = {"documents": list(collection.aggregate(normalize_objectid_inputs(pipeline)))} else: - return error_response("Not a valid operation") + return error_response("Not a valid operation", status_code=400) return success_response(result) - except Exception as e: - print(traceback.format_exc()) - return error_response(e) + except ValueError as exc: + logging.warning(traceback.format_exc()) + return error_response(exc, status_code=400) + + except Exception: + logging.error(traceback.format_exc()) + return error_response("Internal server error", status_code=500) finally: if client: client.close() + +