Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 15 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@
A Python package allowing developers to connect to Dataverse environments for DDL / DML operations.

- Read (SQL) — Execute constrained read-only SQL via the Dataverse Web API `?sql=` parameter. Returns `list[dict]`.
- OData CRUD — Unified methods `create(logical_name, record|records)`, `update(logical_name, id|ids, patch|patches)`, `delete(logical_name, id|ids)` plus `get` with record id or filters.
- OData CRUD — Unified methods `create(logical_name, record|records)`, `update(logical_name, id|ids, patch|patches)`, `delete(logical_name, id|ids)` plus `get` with record id or filters and `delete_async` for better multi-record delete performance.
- Bulk create — Pass a list of records to `create(...)` to invoke the bound `CreateMultiple` action; returns `list[str]` of GUIDs. If any payload omits `@odata.type` the SDK resolves and stamps it (cached).
- Bulk update — Provide a list of IDs with a single patch (broadcast) or a list of per‑record patches to `update(...)`; internally uses the bound `UpdateMultiple` action; returns nothing. Each record must include the primary key attribute when sent to UpdateMultiple.
- Bulk delete - Provide a list of IDs to `delete(...)` or `delete_async(...)`. `delete` internally uses `DeleteMultiple` for elastic tables, for standard tables it is a loop over single-record delete. `delete_async` interally uses async BulkDelete.
- Retrieve multiple (paging) — Generator-based `get(...)` that yields pages, supports `$top` and Prefer: `odata.maxpagesize` (`page_size`).
- Upload files — Call `upload_file(logical_name, ...)` and an upload method will be auto picked (you can override the mode). See https://learn.microsoft.com/en-us/power-apps/developer/data-platform/file-column-data?tabs=sdk#upload-files
- Metadata helpers — Create/inspect/delete tables and create/delete columns (EntityDefinitions + Attributes).
Expand Down Expand Up @@ -39,7 +40,9 @@ Auth:
| `update` | `update(logical_name, list[id], patch)` | `None` | Broadcast; same patch applied to all IDs (UpdateMultiple). |
| `update` | `update(logical_name, list[id], list[patch])` | `None` | 1:1 patches; lengths must match (UpdateMultiple). |
| `delete` | `delete(logical_name, id)` | `None` | Delete one record. |
| `delete` | `delete(logical_name, list[id], use_bulk_delete=True)` | `Optional[str]` | Delete many with async BulkDelete or sequential single-record delete. |
| `delete` | `delete(logical_name, list[id])` | `None` | DeleteMultiple for elastic tables or loops over single-record delete for standard tables. |
| `delete_async` | `delete_async(logical_name, id)` | `str` | Async single-record delete. |
| `delete_async` | `delete_async(logical_name, list[id])` | `str` | Async multi-record delete. |
| `query_sql` | `query_sql(sql)` | `list[dict]` | Constrained read-only SELECT via `?sql=`. |
| `create_table` | `create_table(tablename, schema, solution_unique_name=None)` | `dict` | Creates custom table + columns. Friendly name (e.g. `SampleItem`) becomes schema `new_SampleItem`; explicit schema name (contains `_`) used as-is. Pass `solution_unique_name` to attach the table to a specific solution instead of the default solution. |
| `create_column` | `create_column(tablename, columns)` | `list[str]` | Adds columns using a `{name: type}` mapping (same shape as `create_table` schema). Returns schema names for the created columns. |
Expand All @@ -54,10 +57,10 @@ Auth:

Guidelines:
- `create` always returns a list of GUIDs (1 for single, N for bulk).
- `update` always returns `None`.
- `update` and `delete` always returns `None`.
- Bulk update chooses broadcast vs per-record by the type of `changes` (dict vs list).
- `delete` returns `None` for single-record delete and sequential multi-record delete, and the BulkDelete async job ID for multi-record BulkDelete.
- BulkDelete doesn't wait for the delete job to complete. It returns once the async delete job is scheduled.
- `delete_async` returns the BulkDelete async job ID and doesn't wait for completion.
- It's recommended to use delete_async for multi-record delete for better performance.
- Paging and SQL operations never mutate inputs.
- Metadata lookups for logical name stamping cached per entity set (in-memory).

Expand Down Expand Up @@ -143,8 +146,11 @@ print({"multi_update": "ok"})
# Delete (single)
client.delete("account", account_id)

# Bulk delete (schedules BulkDelete and returns job id)
job_id = client.delete("account", ids)
# Delete (multiple)
client.delete("account", ids)

# Or queue a async bulk delete job
job_id = client.delete_async("account", ids)

# SQL (read-only) via Web API `?sql=`
rows = client.query_sql("SELECT TOP 3 accountid, name FROM account ORDER BY createdon DESC")
Expand Down Expand Up @@ -334,8 +340,8 @@ client.delete_table("SampleItem") # delete table (friendly name or explici

Notes:
- `create` always returns a list of GUIDs (length 1 for single input).
- `update` returns `None`.
- `delete` returns `None` for single-record delete/sequential multi-record delete, and the BulkDelete async job ID for BulkDelete.
- `update` and `delete` returns `None`.
- `delete_async` returns the BulkDelete async job ID.
- Passing a list of payloads to `create` triggers bulk create and returns `list[str]` of IDs.
- `get` supports single record retrieval with record id or paging through result sets (prefer `select` to limit columns).
- For CRUD methods that take a record id, pass the GUID string (36-char hyphenated). Parentheses around the GUID are accepted but not required.
Expand Down
10 changes: 5 additions & 5 deletions examples/quickstart.py
Original file line number Diff line number Diff line change
Expand Up @@ -512,16 +512,16 @@ def run_paging_demo(label: str, *, top: Optional[int], page_size: Optional[int])

# Fire-and-forget bulk delete for the first portion
try:
log_call(f"client.delete('{logical}', <{len(bulk_targets)} ids>, use_bulk_delete=True)")
bulk_job_id = client.delete(logical, bulk_targets)
log_call(f"client.delete_async('{logical}', <{len(bulk_targets)} ids>)")
bulk_job_id = client.delete_async(logical, bulk_targets)
except Exception as ex:
bulk_error = str(ex)

# Sequential deletes for the remainder
try:
log_call(f"client.delete('{logical}', <{len(sequential_targets)} ids>, use_bulk_delete=False)")
log_call(f"client.delete('{logical}', <{len(sequential_targets)} ids>)")
for rid in sequential_targets:
backoff_retry(lambda rid=rid: client.delete(logical, rid, use_bulk_delete=False))
backoff_retry(lambda rid=rid: client.delete(logical, rid))
except Exception as ex:
sequential_error = str(ex)

Expand Down Expand Up @@ -656,4 +656,4 @@ def _ensure_removed():
except Exception as e:
print(f"Delete table failed: {e}")
else:
print({"table_deleted": False, "reason": "user opted to keep table"})
print({"table_deleted": False, "reason": "user opted to keep table"})
53 changes: 42 additions & 11 deletions src/dataverse_sdk/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -208,24 +208,17 @@ def delete(
self,
logical_name: str,
ids: Union[str, List[str]],
use_bulk_delete: bool = True,
) -> Optional[str]:
) -> None:
"""
Delete one or more records by GUID.

:param logical_name: Logical (singular) entity name, e.g. ``"account"``.
:type logical_name: str
:param ids: Single GUID string or list of GUID strings to delete.
:type ids: str or list[str]
:param use_bulk_delete: When ``True`` (default) and ``ids`` is a list, execute the BulkDelete action and
return its async job identifier. When ``False`` each record is deleted sequentially.
:type use_bulk_delete: bool

:raises TypeError: If ``ids`` is not str or list[str].
:raises HttpError: If the underlying Web API delete request fails.

:return: BulkDelete job ID when deleting multiple records via BulkDelete; otherwise ``None``.
:rtype: str or None

Example:
Delete a single record::
Expand All @@ -234,7 +227,7 @@ def delete(

Delete multiple records::

job_id = client.delete("account", [id1, id2, id3])
client.delete("account", [id1, id2, id3])
"""
od = self._get_odata()
if isinstance(ids, str):
Expand All @@ -246,12 +239,50 @@ def delete(
return None
if not all(isinstance(rid, str) for rid in ids):
raise TypeError("ids must contain string GUIDs")
if use_bulk_delete:
return od._delete_multiple(logical_name, ids)
if od._is_elastic_table(logical_name):
od._delete_multiple(logical_name, ids)
return None
for rid in ids:
od._delete(logical_name, rid)
return None

def delete_async(
self,
logical_name: str,
ids: Union[str, List[str]],
) -> str:
"""
Issue an asynchronous BulkDelete job for one or more records.

:param logical_name: Logical (singular) entity name, e.g. ``"account"``.
:type logical_name: str
:param ids: Single GUID string or list of GUID strings to delete.
:type ids: str or list[str]

:raises TypeError: If ``ids`` is not str or list[str].
:raises HttpError: If the BulkDelete request fails.

:return: BulkDelete job identifier, a dummy if ids is empty.
:rtype: str

Example:
Queue a bulk delete::

job_id = client.delete_async("account", [id1, id2, id3])
"""
od = self._get_odata()
if isinstance(ids, str):
return od._delete_async(logical_name, [ids])
elif isinstance(ids, list):
if not ids:
noop_bulkdelete_job_id = "00000000-0000-0000-0000-000000000000"
return noop_bulkdelete_job_id
if not all(isinstance(rid, str) for rid in ids):
raise TypeError("ids must contain string GUIDs")
return od._delete_async(logical_name, ids)
else:
raise TypeError("ids must be str or list[str]")

def get(
self,
logical_name: str,
Expand Down
63 changes: 62 additions & 1 deletion src/dataverse_sdk/odata.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ def __init__(
self._logical_to_entityset_cache: dict[str, str] = {}
# Cache: logical name -> primary id attribute (e.g. accountid)
self._logical_primaryid_cache: dict[str, str] = {}
# Cache: logical name -> whether the table is elastic
self._elastic_table_cache: dict[str, bool] = {}
# Picklist label cache: (logical_name, attribute_logical) -> {'map': {...}, 'ts': epoch_seconds}
self._picklist_label_cache = {}
self._picklist_cache_ttl_seconds = 3600 # 1 hour TTL
Expand Down Expand Up @@ -285,7 +287,35 @@ def _update_by_ids(self, logical_name: str, ids: List[str], changes: Union[Dict[
self._update_multiple(entity_set, logical_name, batch)
return None

def _delete_multiple(
def _delete_multiple(self, logical_name: str, ids: List[str]) -> None:
"""Delete records using the collection-bound DeleteMultiple action.

Parameters
----------
logical_name : str
Singular logical entity name.
ids : list[str]
GUIDs for the records to remove.

Returns
-------
None
No representation is returned.
"""
entity_set = self._entity_set_from_logical(logical_name)
pk_attr = self._primary_id_attr(logical_name)
targets: List[Dict[str, Any]] = []
for rid in ids:
targets.append({
"@odata.type": f"Microsoft.Dynamics.CRM.{logical_name}",
pk_attr: rid,
})
payload = {"Targets": targets}
url = f"{self.api}/{entity_set}/Microsoft.Dynamics.CRM.DeleteMultiple"
self._request("post", url, json=payload)
return None

def _delete_async(
self,
logical_name: str,
ids: List[str],
Expand Down Expand Up @@ -656,6 +686,37 @@ def _entity_set_from_logical(self, logical: str) -> str:
self._logical_primaryid_cache[logical] = primary_id_attr
return es

def _is_elastic_table(self, logical: str) -> bool:
"""Return True when the target table is configured as an elastic table."""
if not logical:
raise ValueError("logical name required")
cached = self._elastic_table_cache.get(logical)
if cached is not None:
return cached
url = f"{self.api}/EntityDefinitions"
logical_escaped = self._escape_odata_quotes(logical)
params = {
"$select": "LogicalName,TableType",
"$filter": f"LogicalName eq '{logical_escaped}'",
}
r = self._request("get", url, params=params)
try:
body = r.json()
items = body.get("value", []) if isinstance(body, dict) else []
except ValueError:
items = []
is_elastic = False
if items:
md = items[0]
if isinstance(md, dict):
table_type = md.get("TableType")
if isinstance(table_type, str):
is_elastic = table_type.strip().lower() == "elastic"
else:
is_elastic = False
self._elastic_table_cache[logical] = is_elastic
return is_elastic

# ---------------------- Table metadata helpers ----------------------
def _label(self, text: str) -> Dict[str, Any]:
lang = int(self.config.language_code)
Expand Down