Skip to content

db-nethra/data-entry-app

BranchesTags

Repository files navigation

Streamlit Databricks App Template

Minimal template for deploying Streamlit apps to Databricks Apps. This example demonstrates a Delta table editor with inline editing capabilities.

What It Does

  • Queries Unity Catalog Delta tables via SQL Warehouse
  • Displays data in an editable Streamlit table
  • Writes updates back to Delta tables
  • Runs as a Databricks App with service principal authentication

Prerequisites

  • Databricks workspace with Apps enabled
  • SQL Warehouse (note the warehouse ID)
  • Unity Catalog table with data
  • Databricks CLI installed and authenticated

Quick Start

1. Configure Environment

Create a .env.local file with your Databricks configuration:

# App name (choose a unique name, lowercase, no spaces)
# This app will be CREATED by ./deploy.sh --create
DATABRICKS_APP_NAME=your-app-name

# Workspace path where source code will be uploaded
# Pattern: /Workspace/Users/<your-email>/<folder-name>
# This folder will be created automatically during deployment
DBA_SOURCE_CODE_PATH=/Workspace/Users/your.email@company.com/your-app-name

# Authentication method (use databricks-cli for profile-based auth)
DATABRICKS_AUTH_TYPE=databricks-cli

# Your Databricks CLI profile name (from ~/.databrickscfg)
# Run `databricks auth login` to create a profile if needed
DATABRICKS_CONFIG_PROFILE=your-profile-name

What gets created automatically:

  • ✅ The Databricks App (when you run ./deploy.sh --create)
  • ✅ The workspace folder at DBA_SOURCE_CODE_PATH
  • ❌ You do NOT need to create anything manually in Databricks first

2. Customize Your App

Edit streamlit_app.py to point to your data:

CATALOG = "your_catalog"
SCHEMA = "your_schema"
TABLE = "your_table"
WAREHOUSE_ID = "your_warehouse_id"

3. Add Python Dependencies

Edit requirements.txt to add new packages:

# Python dependencies for Databricks Apps
# Add new packages here and run ./deploy.sh

streamlit>=1.30.0
pandas>=2.0.0
databricks-sdk>=0.18.0
# Add your packages below:
# plotly>=5.0.0
# scikit-learn>=1.0.0

4. Deploy

First-time deployment (creates the app):

./deploy.sh --create

Subsequent deployments (updates existing app):

./deploy.sh

The --create flag will:

  • Check if the app exists
  • Create it if it doesn't (takes several minutes)
  • Deploy your code to the app

5. Grant Permissions

After deployment, grant your app's service principal:

SQL Warehouse:

  • Navigate to SQL Warehouses → Select your warehouse → Permissions
  • Add service principal (named app-XXXXX <your-app-name>) with Can Use permission

Unity Catalog:

  • Grant at catalog level OR specific schema/table level:
-- Option 1: Catalog-level (easiest)
GRANT USAGE ON CATALOG your_catalog TO `app-XXXXX your-app-name`;
GRANT USAGE ON SCHEMA your_catalog.your_schema TO `app-XXXXX your-app-name`;
GRANT SELECT ON TABLE your_catalog.your_schema.your_table TO `app-XXXXX your-app-name`;
GRANT MODIFY ON TABLE your_catalog.your_schema.your_table TO `app-XXXXX your-app-name`;

-- Option 2: Grant ALL PRIVILEGES at catalog level
GRANT ALL PRIVILEGES ON CATALOG your_catalog TO `app-XXXXX your-app-name`;

SQL Warehouse Resource (via UI):

  • Go to your Databricks App settings → Resources
  • Add SQL Warehouse with "Can use" permission

Local Development vs Deployed App

Local Testing

Run locally for testing before deployment:

streamlit run streamlit_app.py --server.port=8501

Why port 8501? This is Streamlit's default development port. Use any available port locally - it doesn't matter for local testing.

Local authentication: Uses your Databricks CLI profile from ~/.databrickscfg. Make sure to use WorkspaceClient(profile="your-profile-name") in local mode.

Deployed App

When deployed to Databricks Apps, key differences:

  1. Port: MUST use ${DATABRICKS_APP_PORT} environment variable (typically 8000)
    • This is set in app.yaml: --server.port=${DATABRICKS_APP_PORT:-8000}
    • Databricks Apps routes traffic to this port
  2. Authentication: Uses app service principal automatically, not user credentials
  3. WorkspaceClient: Use WorkspaceClient() with NO profile parameter

Common Gotchas

Port Mismatch:

  • Symptom: "App Not Available" error
  • Fix: App must listen on ${DATABRICKS_APP_PORT} (set by Databricks Apps)
  • Correct: --server.port=${DATABRICKS_APP_PORT:-8000}
  • Wrong: --server.port=8080

Permission Errors:

  • Symptom: Empty tables or "no data" errors
  • Fix: Ensure service principal has SQL Warehouse + Unity Catalog permissions
  • Check app SP name: databricks apps get <app-name> --output json | grep service_principal_name

Authentication:

  • Local: WorkspaceClient(profile="your-profile") works
  • Deployed: WorkspaceClient() required (no profile parameter)

Customizing Your App

Adding Your Company Logo

Add a logo to the Streamlit sidebar:

import streamlit as st
from PIL import Image

# At the top of your app
st.set_page_config(
    page_title="Your Company - Loan Exceptions",
    page_icon="🏢",  # Or use your logo emoji
    layout="wide"
)

# Add logo in sidebar
with st.sidebar:
    # Option 1: Load from file (upload logo.png to your workspace)
    logo = Image.open("logo.png")
    st.image(logo, width=200)

    # Option 2: Load from URL
    st.image("https://your-company.com/logo.png", width=200)

    st.title("Loan Exceptions")

Customizing Colors and Theme

Create a .streamlit/config.toml file in your app directory:

[theme]
primaryColor = "#FF6B6B"        # Your brand color (buttons, links)
backgroundColor = "#FFFFFF"     # Main background
secondaryBackgroundColor = "#F0F2F6"  # Sidebar, inputs
textColor = "#262730"           # Main text
font = "sans serif"             # Options: "sans serif", "serif", "monospace"

# Example: Corporate blue theme
# primaryColor = "#1E40AF"
# backgroundColor = "#FFFFFF"
# secondaryBackgroundColor = "#EFF6FF"
# textColor = "#1F2937"

Upload the config file to your workspace:

# Add to .streamlit/config.toml, then redeploy
./deploy.sh

Helper Functions for Common Tasks

1. Format Currency Values

def format_currency(value):
    """Format number as USD currency"""
    return f"${value:,.2f}"

# Usage in dataframe
df['loan_amount_display'] = df['loan_amount'].apply(format_currency)
st.dataframe(df)

2. Add Color-Coded Status Badges

def get_status_color(status):
    """Return color for status badge"""
    colors = {
        "Open": "🔴",
        "In Progress": "🟡",
        "Cleared": "🟢",
        "Closed": "⚫"
    }
    return colors.get(status, "⚪")

# Usage
df['status_display'] = df['status'].apply(lambda x: f"{get_status_color(x)} {x}")

3. Add Export to Excel/CSV

import pandas as pd
from io import BytesIO

def export_to_excel(df):
    """Export dataframe to Excel file"""
    output = BytesIO()
    with pd.ExcelWriter(output, engine='openpyxl') as writer:
        df.to_excel(writer, index=False, sheet_name='Exceptions')
    return output.getvalue()

# Usage in your app
if st.button("📥 Export to Excel"):
    excel_data = export_to_excel(df)
    st.download_button(
        label="Download Excel",
        data=excel_data,
        file_name="loan_exceptions.xlsx",
        mime="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
    )

4. Add Filtering Sidebar

def add_filter_sidebar(df):
    """Add filter controls to sidebar"""
    with st.sidebar:
        st.header("Filters")

        # Status filter
        statuses = ["All"] + list(df['status'].unique())
        selected_status = st.selectbox("Status", statuses)

        # Severity filter
        severities = ["All"] + list(df['severity'].unique())
        selected_severity = st.selectbox("Severity", severities)

        # Date range
        date_range = st.date_input(
            "Date Range",
            value=(df['created_date'].min(), df['created_date'].max())
        )

    # Apply filters
    filtered_df = df.copy()
    if selected_status != "All":
        filtered_df = filtered_df[filtered_df['status'] == selected_status]
    if selected_severity != "All":
        filtered_df = filtered_df[filtered_df['severity'] == selected_severity]

    return filtered_df

5. Add Metrics Dashboard

def show_metrics(df):
    """Display key metrics in columns"""
    col1, col2, col3, col4 = st.columns(4)

    with col1:
        st.metric(
            label="Total Exceptions",
            value=len(df),
            delta=f"+{len(df[df['status']=='Open'])} Open"
        )

    with col2:
        avg_resolution_days = (df['updated_date'] - df['created_date']).dt.days.mean()
        st.metric(
            label="Avg Resolution Time",
            value=f"{avg_resolution_days:.1f} days"
        )

    with col3:
        high_severity = len(df[df['severity'] == 'High'])
        st.metric(
            label="High Severity",
            value=high_severity,
            delta=f"{(high_severity/len(df)*100):.1f}%"
        )

    with col4:
        cleared_rate = len(df[df['status'] == 'Cleared']) / len(df) * 100
        st.metric(
            label="Clearance Rate",
            value=f"{cleared_rate:.1f}%"
        )

Modifying the Data Model

To add new columns to your Delta table:

# Run this once to add a new column
from databricks.sdk import WorkspaceClient

w = WorkspaceClient(profile="your-profile")

query = """
ALTER TABLE your_catalog.your_schema.your_table
ADD COLUMN new_column_name STRING
"""

response = w.statement_execution.execute_statement(
    warehouse_id="your_warehouse_id",
    statement=query,
    catalog="your_catalog",
    schema="your_schema"
)

Then update your load_data() query and editable columns list in streamlit_app.py:

# Add to SELECT statement
query = f"""
    SELECT
        exception_id,
        loan_id,
        ...,
        new_column_name  -- Add your new column
    FROM {CATALOG}.{SCHEMA}.{TABLE}
"""

# Add to editable columns list
editable_columns = [
    'loan_id',
    'exception_type',
    'severity',
    'status',
    'owner',
    'description',
    'additional_comments',
    'new_column_name'  -- Add your new column
]

Development Workflow

  1. Edit code: Modify streamlit_app.py or add dependencies to requirements.txt
  2. Test locally: streamlit run streamlit_app.py --server.port=8501
  3. Deploy: ./deploy.sh to push changes to Databricks Apps
  4. Monitor: Visit https://<your-app-url>/logz to check deployment logs

File Structure

├── streamlit_app.py          # Your Streamlit application
├── app.yaml                   # Databricks Apps configuration
├── requirements.txt           # Python dependencies (edit directly)
├── deploy.sh                  # Deployment script
├── .env.local                 # Local environment config (not committed)
├── dba_logz.py               # Helper: View deployment logs
└── dba_client.py             # Helper: Test authenticated requests

Key files to customize:

  • streamlit_app.py - Your app logic, UI, and queries
  • requirements.txt - Add Python packages here
  • .streamlit/config.toml - Theme and styling (create if needed)
  • app.yaml - App startup command and port config

Useful Commands

# Deploy app
./deploy.sh

# Check app status
databricks apps get <app-name>

# View app service principal
databricks apps get <app-name> --output json | grep service_principal_name

# Test deployed app endpoint
curl https://<app-url>

Debugging

Use the included helper scripts:

# View deployment logs (requires OAuth in browser)
uv run python dba_logz.py --app_url https://<your-app-url> --duration 60

# Test authenticated requests
uv run python dba_client.py https://<your-app-url> /health

License

See LICENSE.md

About

data entry app demo

Resources

Readme

License

View license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages

Generated from databricks-solutions/claude-databricks-app-template