Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 4 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Repository Guidelines

These guidelines help contributors work effectively on this repository. We gratefully acknowledge [mcp-postgres](https://github.com/gldc/mcp-postgres) as the chief inspiration for the MCP server function and this document.
These guidelines help contributors, human and otherwise, understand and work effectively on this repository.

We also encourage reading [`docs/developer_guide.md`](/docs/developer_guide.md) for further useful information.
We also encourage reading [`docs/developer_guide.md`](/docs/developer_guide.md) for further useful information. For deeper understanding of the "brains" of `stackql`, it is worthwhile to consult [the `AGENTS.md` file of `any-sdk`](https://github.com/stackql/any-sdk/blob/main/AGENTS.md) and linked documents from there.


## Project Structure & Module Organization
Expand All @@ -18,19 +18,15 @@ We also encourage reading [`docs/developer_guide.md`](/docs/developer_guide.md)

## Build, Test, and Development Commands

- Create env: `python -m venv .venv && source .venv/bin/activate`
- Install deps: `pip install -r requirements.txt`
- Run server (no DB): `python postgres_server.py`
- Run with DB: `POSTGRES_CONNECTION_STRING="postgresql://user:pass@host:5432/db" python postgres_server.py`
- Docker build/run: `docker build -t mcp-postgres .` then `docker run -e POSTGRES_CONNECTION_STRING=... -p 8000:8000 mcp-postgres`
Please refer to [the developer guide](/docs/developer_guide.md), [the testing summary document](/test/README.md), [the robot testing document](/test/robot/README.md), and possibly most useful of all, [the doco explaining testing with mocks](/test/python/stackql_test_tooling/flask/README.md). For CI in the wild, please see [`.github/workflows/build.yml`](/.github/workflows/build.yml).

## Coding Style & Naming Conventions

- Publish and program to abstractions.

## Testing Guidelines

- Black box regression tests are effectively mandatory. The canaonical ones reside in [`test/robot/functional`](/test/robot/functional).
- Black box regression tests are effectively mandatory. The canaonical ones reside in [`test/robot`](/test/robot/README.md).

## Tools & Resources

Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@

## About The Project

[__StackQL__][homepage] is an open-source project built with Golang that allows you to create, modify and query the state of services and resources across different cloud and SaaS providers (Google, AWS, Azure, Okta, GitHub, etc.) using SQL semantics
[__StackQL__][homepage] is an open-source project built with Golang that allows you to create, modify and query the state of services and resources across different local and remote interfaces, using SQL semantics. Such interfaces canonically include, but are not limited to, cloud and SaaS providers (Google, AWS, Azure, Okta, GitHub, etc.).
<br />
<br />

Expand All @@ -102,9 +102,9 @@

StackQL is a standalone application that can be used in client mode (via __`exec`__ or __`shell`__) or accessed via a Postgres wire protocol client (`psycopg2`, etc.) using server mode (__`srv`__).

StackQL parses SQL statements and transpiles them into API requests to the cloud provider. The API calls are then executed and the results are returned to the user.
StackQL parses SQL statements and transpiles them into API requests to the (cloud) resource provider. The API calls are then executed and the results are returned to the user.

StackQL provider definitions are defined in OpenAPI extensions to the providers specification. These definitions are then used to generate the SQL schema and the API client. The source for the provider definitions are stored in the [__StackQL Registry__][registry].
StackQL provider interfaces are canonically defined in OpenAPI extensions to the providers' specification. These definitions are then used to generate the SQL schema and the API client. The source for the provider definitions are stored in the [__StackQL Registry__][registry]. The semantics of provider interactions are defined in [our `any-sdk` library](https://github.com/stackql/any-sdk). For more detail on nuts and bolts, please see [the local `AGENTS.md` file](/AGENTS.md) and [that of `any-sdk`](https://github.com/stackql/any-sdk/blob/main/AGENTS.md).

<details>
<summary><b>StackQL Context Diagram</b></summary>
Expand Down
2 changes: 2 additions & 0 deletions cicd/scripts/testing-env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ googleCredentialsFilePath="${workspaceFolder}/test/assets/credentials/dummy/goog

export stackqlMockedRegistryStr="{ \"url\": \"file://${workspaceFolder}/test/registry-mocked\", \"localDocRoot\": \"${workspaceFolder}/test/registry-mocked\", \"verifyConfig\": { \"nopVerify\": true } }"

export stackqlTestRegistryStr="{ \"url\": \"file://${workspaceFolder}/test/registry\", \"localDocRoot\": \"${workspaceFolder}/test/registry\", \"verifyConfig\": { \"nopVerify\": true } }"

export stackqlAuthStr="{\"google\": {\"credentialsfilepath\": \"${googleCredentialsFilePath}\", \"type\": \"service_account\"}, \"okta\": {\"credentialsenvvar\": \"OKTA_SECRET_KEY\", \"type\": \"api_key\"}, \"aws\": {\"type\": \"aws_signing_v4\", \"credentialsfilepath\": \"/Users/admin/stackql/stackql-devel/test/robot/functional/../../../test/assets/credentials/dummy/aws/functional-test-dummy-aws-key.txt\", \"keyID\": \"NON_SECRET\"}, \"github\": {\"type\": \"basic\", \"credentialsenvvar\": \"GITHUB_SECRET_KEY\"}, \"k8s\": {\"credentialsenvvar\": \"K8S_SECRET_KEY\", \"type\": \"api_key\", \"valuePrefix\": \"Bearer \"}, \"azure\": {\"type\": \"api_key\", \"valuePrefix\": \"Bearer \", \"credentialsenvvar\": \"AZ_ACCESS_TOKEN\"}, \"sumologic\": {\"type\": \"basic\", \"credentialsenvvar\": \"SUMO_CREDS\"}, \"digitalocean\": {\"type\": \"bearer\", \"username\": \"myusername\", \"password\": \"mypassword\"}}"


Expand Down
12 changes: 0 additions & 12 deletions docs/integration_testing.md

This file was deleted.

4 changes: 2 additions & 2 deletions test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@
Offline invocations of `stackql` are assessed against expected responses, through:

1. the functionality of [/test/deprecated/python/main.py](/test/deprecated/python/main.py).
2. [robot tests in /test/functional](/test/functional)
2. robot tests, [per this document](/test/robot/README.md).

(1) is deprecated and will be entirely migrated to (2).

These tests are run during the build process:
- locally as per [/README.md#build](/README.md#build)
- in github actions based CICD as per [/.github/workflows/go.yml](/.github/workflows/go.yml).
- in github actions based CICD as per [/.github/workflows/go.yml](/.github/workflows/build.yml).

## Unit tests using standard golang approaches

Expand Down
12 changes: 2 additions & 10 deletions test/robot/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,4 @@


## TODO

- [ ] Source registry files from other repository, where possible.
- [x] "reuired" string.
- [x] clean up this readme.
- [x] integration tests for different registry configurations.
- [x] fix library lifetimes issue; as per https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#creating-test-library-class-or-module.


## Distributed testing

It is convenient for development- and release-critical robot tests to reside in this repository, because it accelerates the
Expand Down Expand Up @@ -107,6 +97,8 @@ $v1="SELECT i.zone, i.name, i.machineType, i.deletionProtection, '[{""""""subnet

## Session testing for Server and Shell

**NOTE**: This is deprecated; robot is the way formward. Do **not** add new tests with this pattern.

Basic idea is have python start a session, run commands with result verification and then terminate. Probably custom library(ies).

Library would do something like the below adaptation of [this example from stack overflow](https://stackoverflow.com/questions/19880190/interactive-input-output-using-python):
Expand Down
Loading