diff --git a/docs/how-tos/datacoves/how_to_projects/how_to_configure_aws_secrets_manager.md b/docs/how-tos/datacoves/how_to_projects/how_to_configure_aws_secrets_manager.md index bfb1d849..f50461f8 100644 --- a/docs/how-tos/datacoves/how_to_projects/how_to_configure_aws_secrets_manager.md +++ b/docs/how-tos/datacoves/how_to_projects/how_to_configure_aws_secrets_manager.md @@ -7,9 +7,9 @@ sidebar_position: 50 ## Table of Contents - [Prereqs](#prereqs) - [Create your Secret in AWS Secrets Manager](#create-your-secret-in-aws-secrets-manager) -- [Configure your Secrets Backend in Project settings](#configure-your-secrets-backend-in-project-settings) +- [Configure your Secrets Backend](#configure-your-secrets-backend) - [Project-level configuration](#project-level-configuration) - - [Environment-level override](#environment-level-override) + - [Environment-level configuration](#environment-level-configuration) ## Prereqs diff --git a/docs/how-tos/vs-code/environment-variables.md b/docs/how-tos/vs-code/environment-variables.md index 8c69e545..6376b4aa 100644 --- a/docs/how-tos/vs-code/environment-variables.md +++ b/docs/how-tos/vs-code/environment-variables.md @@ -73,6 +73,12 @@ Creating variables is simple. Add the key following the correct format and add t ![Create variables](assets/env_var_environment_create.png) +:::tip +Some features read a value from an environment variable you set here. For example, the [GitHub MCP +server](/docs/how-tos/vs-code/mcp/github) reads a personal access token from a user-level variable named +`GITHUB_PERSONAL_ACCESS_TOKEN`. Its value is automatically masked because the name contains `token`. +::: + ## Sensitive Variable Masking Datacoves automatically masks the values of environment variables whose names contain certain keywords. This helps prevent accidental exposure of secrets in the UI. diff --git a/docs/how-tos/vs-code/external-ai-tools/github-copilot.md b/docs/how-tos/vs-code/external-ai-tools/github-copilot.md new file mode 100644 index 00000000..553dd0d0 --- /dev/null +++ b/docs/how-tos/vs-code/external-ai-tools/github-copilot.md @@ -0,0 +1,31 @@ +--- +title: GitHub Copilot +sidebar_position: 3 +--- + +# GitHub Copilot + +[GitHub Copilot](https://github.com/features/copilot) is built into the Datacoves VS Code +workspace as the GitHub Copilot and Copilot Chat extensions. + +## Prerequisites + +- A GitHub account with an active GitHub Copilot subscription. +- GitHub Copilot enabled in your Datacoves environment. An administrator enables it under + **Admin > Environments > _your environment_ > AI Tools > AI Extensions**. + +## Login + +1. Open your Datacoves VS Code workspace. +2. Open the Copilot Chat view, or click the Copilot icon in the status bar. +3. Choose to sign in and follow the GitHub authentication prompt in your browser. +4. Once authorized, Copilot is ready in both inline suggestions and chat. + +## MCP servers + +GitHub Copilot can use the Datacoves [MCP servers](/docs/how-tos/vs-code/mcp) (GitHub, Airflow, +Grafana) when they are enabled for your environment. + +## Learn more + +- [GitHub Copilot documentation](https://docs.github.com/en/copilot) diff --git a/docs/how-tos/vs-code/external-ai-tools/snowflake-cortex.md b/docs/how-tos/vs-code/external-ai-tools/snowflake-cortex.md new file mode 100644 index 00000000..f753d874 --- /dev/null +++ b/docs/how-tos/vs-code/external-ai-tools/snowflake-cortex.md @@ -0,0 +1,33 @@ +--- +title: Snowflake Cortex +sidebar_position: 2 +--- + +# Snowflake Cortex + +[Snowflake Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code-cli) is +Snowflake's AI coding assistant, available as a CLI tool in Datacoves. + +## Prerequisites + +- A Snowflake-based (`dbt-snowflake`) environment. Cortex is only available on Snowflake + environments. +- Cortex enabled in your Datacoves environment. Contact + [Datacoves support](mailto:support@datacoves.com) to enable it, or an administrator can turn it + on under **Admin > Environments > _your environment_ > AI Tools**. + +## Login and usage + +Cortex authenticates against Snowflake and is driven from a terminal in your Datacoves VS Code +workspace. See +[Snowflake's Cortex Code CLI documentation](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code-cli) +for the authentication and usage steps. + +## MCP servers + +Cortex can use the Datacoves [MCP servers](/docs/how-tos/vs-code/mcp) (GitHub, Airflow, Grafana) +when they are enabled for your environment. + +## Learn more + +- [Snowflake Cortex Code documentation](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code-cli) diff --git a/docs/how-tos/vs-code/mcp/README.md b/docs/how-tos/vs-code/mcp/README.md new file mode 100644 index 00000000..e47cc062 --- /dev/null +++ b/docs/how-tos/vs-code/mcp/README.md @@ -0,0 +1,54 @@ +--- +title: MCP Servers +sidebar_position: 46 +--- + +# MCP Servers + +Datacoves can connect your AI coding assistants to live data systems through +[MCP (Model Context Protocol)](https://modelcontextprotocol.io/) servers. An MCP server gives an +AI tool structured, read access to an external system so it can answer questions grounded in your +real data instead of guessing. + +Datacoves ships three MCP servers: + +| Server | What it exposes | User setup | +|--------|-----------------|------------| +| [GitHub](/docs/how-tos/vs-code/mcp/github) | Your repositories, pull requests, issues, and CI checks | A GitHub personal access token | +| [Airflow](/docs/how-tos/vs-code/mcp/airflow) | Your DAGs, runs, and task logs (read-only) | None | +| [Grafana (Prometheus & Loki)](/docs/how-tos/vs-code/mcp/grafana) | Metrics and logs from your environment | None | + +Once enabled, these servers are available to every AI tool in your workspace: + +- Datacoves Copilot +- GitHub Copilot +- [OpenAI Codex](/docs/how-tos/vs-code/external-ai-tools/openai-codex) +- [Snowflake Cortex](/docs/how-tos/vs-code/external-ai-tools/snowflake-cortex) + +## Enabling MCP servers + +MCP servers are toggled per environment by an administrator. + +:::note +You need admin access to enable MCP servers. Go to **Admin > Environments**, edit the +environment, open the **AI Tools** tab, and turn on the servers you want under **MCP Servers**. +::: + +![AI Tools tab](./assets/ai-tools-mcp-toggles.png) + +After a server is enabled, it appears automatically in each AI tool the next time the workspace +starts. No further configuration is needed in the tools themselves. + +:::tip +The **Grafana (Prometheus & Loki)** toggle only appears when the **observability stack** is +enabled for your cluster. Contact [Datacoves support](mailto:support@datacoves.com) if you do not +see it. +::: + +## What you can ask + +Each server page includes example prompts. A few to get started: + +- **GitHub:** "Check my last pull request and explain why the CI check failed." +- **Airflow:** "Check the log of the task that failed in my last DAG run and recommend a fix." +- **Grafana:** "Query the metrics and logs for my environment and summarize recent failures." diff --git a/docs/how-tos/vs-code/mcp/airflow.md b/docs/how-tos/vs-code/mcp/airflow.md new file mode 100644 index 00000000..2340c425 --- /dev/null +++ b/docs/how-tos/vs-code/mcp/airflow.md @@ -0,0 +1,44 @@ +--- +title: Airflow +sidebar_position: 2 +--- + +# Airflow MCP Server + +The Airflow MCP server lets your AI tools read your DAGs, runs, and task logs, so you can ask the +AI to investigate a failed task and recommend a fix without leaving VS Code. + +It is backed by [Astronomer's Airflow MCP server](https://github.com/astronomer/agents/tree/main/astro-airflow-mcp) +and works with both Airflow 2 and Airflow 3. + +## Prerequisites + +- The **Airflow** MCP server enabled for your environment (see [Enabling](#enabling)). +- Airflow enabled and running for your environment. + +## How it works + +- **No setup required.** The server authenticates as **you**, using your Datacoves identity, so it + only sees what your Airflow permissions allow. +- **Read-only by default.** The server is configured for read access, so the AI can inspect DAGs, + runs, and logs but cannot trigger, pause, or change anything. +- Works with both your team's Airflow and, when enabled, your personal **My Airflow** instance. + +## Enabling + +:::note +An administrator enables the server in **Admin > Environments > _your environment_ > AI Tools > +MCP Servers** by turning on **Airflow**. +::: + +## Use it + +Start (or restart) your workspace so the AI tools pick up the server, then ask, for example: + +> Check the log of the task that failed in my last DAG run and recommend a fix. + +> Which DAGs failed today, and what was the error? + +## Learn more + +- [Astronomer Airflow MCP server](https://github.com/astronomer/agents/tree/main/astro-airflow-mcp) diff --git a/docs/how-tos/vs-code/mcp/assets/ai-tools-mcp-toggles.png b/docs/how-tos/vs-code/mcp/assets/ai-tools-mcp-toggles.png new file mode 100644 index 00000000..13b7ef9b Binary files /dev/null and b/docs/how-tos/vs-code/mcp/assets/ai-tools-mcp-toggles.png differ diff --git a/docs/how-tos/vs-code/mcp/assets/github-pat-env-var.png b/docs/how-tos/vs-code/mcp/assets/github-pat-env-var.png new file mode 100644 index 00000000..0eb28090 Binary files /dev/null and b/docs/how-tos/vs-code/mcp/assets/github-pat-env-var.png differ diff --git a/docs/how-tos/vs-code/mcp/github.md b/docs/how-tos/vs-code/mcp/github.md new file mode 100644 index 00000000..e71b0d3b --- /dev/null +++ b/docs/how-tos/vs-code/mcp/github.md @@ -0,0 +1,63 @@ +--- +title: GitHub +sidebar_position: 1 +--- + +# GitHub MCP Server + +The GitHub MCP server lets your AI tools read your GitHub repositories, pull requests, issues, and +CI checks, so you can ask questions like why a check failed and get an answer based on the real +pull request. + +It is backed by [GitHub's official MCP server](https://github.com/github/github-mcp-server). + +## Prerequisites + +- The **GitHub** MCP server enabled for your environment (see [Enabling](#enabling)). +- A GitHub [personal access token](https://github.com/settings/tokens) with access to the + repositories you want the AI to read. + +## Provide your GitHub token + +The GitHub MCP server acts as **you**, using your own personal access token. You provide it as a +VS Code environment variable, so it is never shared with other users. + +### Step 1: Create a personal access token + +In GitHub, create a [personal access token](https://github.com/settings/tokens) (classic or +fine-grained) with at least `repo` scope. Copy the token, you will not be able to see it again. + +### Step 2: Add it as an environment variable + +Add the token as a [user-level VS Code environment variable](/docs/how-tos/vs-code/environment-variables) +named: + +``` +GITHUB_PERSONAL_ACCESS_TOKEN +``` + +![Add the GitHub token](./assets/github-pat-env-var.png) + +:::tip +The value is automatically masked in the UI because the variable name contains `token`. +::: + +## Enabling + +:::note +An administrator enables the server in **Admin > Environments > _your environment_ > AI Tools > +MCP Servers** by turning on **GitHub**. +::: + +## Use it + +Start (or restart) your workspace so the AI tools pick up the server, then ask, for example: + +> Check my last pull request, figure out why the CI check failed, and recommend a fix. + +> List my open pull requests in this repository. + +## Learn more + +- [GitHub MCP server](https://github.com/github/github-mcp-server) +- [Managing personal access tokens](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) diff --git a/docs/how-tos/vs-code/mcp/grafana.md b/docs/how-tos/vs-code/mcp/grafana.md new file mode 100644 index 00000000..48803c45 --- /dev/null +++ b/docs/how-tos/vs-code/mcp/grafana.md @@ -0,0 +1,44 @@ +--- +title: Grafana (Prometheus & Loki) +sidebar_position: 3 +--- + +# Grafana MCP Server + +The Grafana MCP server lets your AI tools query the metrics (Prometheus) and logs (Loki) for your +environment, so you can ask the AI to summarize recent failures or look up a specific log. + +It is backed by [Grafana's MCP server](https://github.com/grafana/mcp-grafana). + +## Prerequisites + +- The **observability stack** enabled for your cluster. The Grafana MCP toggle only appears when + it is. Contact [Datacoves support](mailto:support@datacoves.com) if you need it enabled. +- The **Grafana (Prometheus & Loki)** MCP server enabled for your environment (see + [Enabling](#enabling)). + +## How it works + +**No setup required.** Datacoves provisions a per-environment service account for the server, so +there is nothing for you to configure. The server connects to your environment's Grafana, which +already has the Prometheus and Loki data sources wired up. + +## Enabling + +:::note +An administrator enables the server in **Admin > Environments > _your environment_ > AI Tools > +MCP Servers** by turning on **Grafana (Prometheus & Loki)**. +::: + +## Use it + +Start (or restart) your workspace so the AI tools pick up the server, then ask, for example: + +> Query the metrics and logs for my environment and summarize recent failures. + +> Show me the Loki logs for my Airflow workers in the last hour. + +## Learn more + +- [Grafana MCP server](https://github.com/grafana/mcp-grafana) +- [Grafana dashboards in Datacoves](/docs/how-tos/datacoves/metrics-and-logs/grafana) diff --git a/docs/reference/admin-menu/environments.md b/docs/reference/admin-menu/environments.md index a6b6d0c0..ed125795 100644 --- a/docs/reference/admin-menu/environments.md +++ b/docs/reference/admin-menu/environments.md @@ -25,3 +25,21 @@ On the Environments landing page you can see a list of environments associated w For each environment we can see the associated project, the name of the environment to be displayed on the landing page, and the number of associated service connections. Each row contains 2 action buttons, Edit and Delete. + +## AI Tools + +When editing an environment that has VS Code enabled, the **AI Tools** tab controls the AI +assistants and the data systems they can reach. It has three sections: + +- **AI Extensions** - enable or disable the built-in AI extensions, such as Datacoves Copilot and + GitHub Copilot. +- **AI Tools** - enable or disable the AI CLI tools, such as OpenAI Codex and Snowflake Cortex + (Cortex is only shown on Snowflake environments). +- **MCP Servers** - enable or disable the [MCP servers](/docs/how-tos/vs-code/mcp) (GitHub, + Airflow, and Grafana) that give the AI tools read access to your repositories, DAGs, and + metrics and logs. + +:::note +The **Grafana (Prometheus & Loki)** MCP server only appears when the observability stack is +enabled for the cluster. +:::