diff --git a/docs/code-navigation/explanations/uploads.mdx b/docs/code-navigation/explanations/uploads.mdx index 8436af333..29fcb39f7 100644 --- a/docs/code-navigation/explanations/uploads.mdx +++ b/docs/code-navigation/explanations/uploads.mdx @@ -4,7 +4,7 @@ How Code Graph indexers analyze code and generate and index file.

-[Code graph indexers](/code-navigation/writing-an-indexer) analyze source code and generate an index file, which is subsequently [uploaded to a Sourcegraph instance](/code-navigation/how-to/index-other-languages#upload-scip-data) using [Sourcegraph CLI](/cli/) for processing. Once processed, this data becomes available for [precise code navigation queries](/code-navigation/precise-code-navigation). +[Code graph indexers](/code-navigation/writing-an-indexer) analyze source code and generate an index file, which is subsequently [uploaded to a Sourcegraph instance](/code-navigation/how-to/adding-scip-to-workflows) using [Sourcegraph CLI](/cli/) for processing. Once processed, this data becomes available for [precise code navigation queries](/code-navigation/precise-code-navigation). ## Lifecycle of an upload diff --git a/docs/code-navigation/how-to/adding-scip-to-workflows.mdx b/docs/code-navigation/how-to/adding-scip-to-workflows.mdx index 88dc34ba9..513733013 100644 --- a/docs/code-navigation/how-to/adding-scip-to-workflows.mdx +++ b/docs/code-navigation/how-to/adding-scip-to-workflows.mdx @@ -1,12 +1,12 @@ # Adding precise code navigation to CI/CD workflows -## Language-specific guides - -We are working on creating language specific guides for use with different indexers, so make sure to check for the documentation for your language! If there isn't a guide for your language, this general guide will help you through the precise code navigation setup process. + + This page covers **general CI/CD patterns** — container usage, sub-projects, custom build environments, and upload frequency. For language-specific indexer installation and usage, start with the [language guide for your codebase](/code-navigation/precise-code-navigation#setting-up-code-navigation-for-your-codebase). + ## Benefits of CI integration -Setting up a source code indexing job in your CI build provides you with fast code navigation that gives you more control on when source code gets indexed, and ensures accuracy of your code navigation by keeping in sync with changes in your repository. Due to the large number of CI frameworks and languages we may not have specific documentation for your use case. Feel free to [contact us](https://help.sourcegraph.com/hc/en-us/requests/new) if you're having difficulties and we can help troubleshoot your setup. +Setting up a [SCIP](https://github.com/sourcegraph/scip) indexing job in your CI build gives you more control over when source code gets indexed and ensures accuracy of your code navigation by keeping in sync with changes in your repository. Due to the large number of CI frameworks and languages we may not have specific documentation for your use case. Feel free to [contact us](https://help.sourcegraph.com/hc/en-us/requests/new) if you're having difficulties and we can help troubleshoot your setup. ## Using indexer containers @@ -19,12 +19,12 @@ Some indexers will work out of the box by just running them in your project root ```yaml jobs: scip-go: - #this line will prevent forks of this repo from uploading lsif indexes + # Prevent forks of this repo from uploading SCIP indexes if: github.repository == '' runs-on: ubuntu-latest container: sourcegraph/scip-go:latest steps: - - uses: actions/checkout@v1 + - uses: actions/checkout@v4 - name: Generate index run: scip-go - name: Install src CLI @@ -49,7 +49,7 @@ jobs: runs-on: ubuntu-latest container: sourcegraph/scip-go:latest steps: - - uses: actions/checkout@v1 + - uses: actions/checkout@v4 - name: Generate index working-directory: backend/ run: scip-go @@ -78,12 +78,12 @@ This second step is easy in GitHub actions because our container can be used as ```yaml jobs: scip-typescript: - # this line will prevent forks of this repo from uploading scip indexes + # Prevent forks of this repo from uploading SCIP indexes if: github.repository == '' runs-on: ubuntu-latest container: my-awesome-container steps: - - uses: actions/checkout@v1 + - uses: actions/checkout@v4 - name: Install dependencies run: - name: Generate index @@ -98,9 +98,9 @@ jobs: args: src code-intel upload -github-token=${{ secrets.GITHUB_TOKEN }} -ignore-upload-failure ``` -### Circle CI Examples +### CircleCI examples -Certain frameworks like Circle CI may require you to explicitly cache artifacts between jobs. In CircleCI this might look like the following: +Certain frameworks like CircleCI may require you to explicitly cache artifacts between jobs. This might look like the following: ```yaml version: 2.1 @@ -142,7 +142,7 @@ workflows: ## CI from scratch -If you're indexing a language we haven't documented yet in our [language-specific guides](/code-navigation/precise-code-navigation#setting-up-code-navigation-for-your-codebase), follow the instructions in this section. We want to have containers available for every language with a robust indexer, so please [contact](https://help.sourcegraph.com/hc/en-us/requests/new) to let us know we're missing one. +If you're indexing a language that doesn't have a dedicated guide in our [language-specific guides](/code-navigation/precise-code-navigation#setting-up-code-navigation-for-your-codebase), follow the instructions in this section. We want to have containers available for every language with a robust indexer, so please [contact us](https://help.sourcegraph.com/hc/en-us/requests/new) to let us know we're missing one. ### Set up your CI machines @@ -153,8 +153,8 @@ Your CI machines will need two command-line tools installed. Depending on your b ### Add steps to your CI -1. **Generate an index** for a project within your repository by running the indexer in the project directory (consult your indexer's docs). -1. **[Upload that generated index](/code-navigation/precise-code-navigation#setting-up-code-navigation-for-your-codebase)** to your Sourcegraph instance. +1. **Generate a SCIP index** for a project within your repository by running the indexer in the project directory (consult your indexer's docs). +1. **[Upload the generated index](/code-navigation/precise-code-navigation#setting-up-code-navigation-for-your-codebase)** to your Sourcegraph instance. ## Recommended upload frequency @@ -169,3 +169,20 @@ With periodic jobs, you should still receive precise code navigation on non-inde Indexes can be uploaded to a self-hosted Sourcegraph instance or to [Sourcegraph.com](https://sourcegraph.com). Using the [Sourcegraph.com](https://sourcegraph.com) endpoint will surface code navigation for your public repositories directly on GitHub via the [Sourcegraph browser extension](/integration/browser-extension) and at `https://sourcegraph.com/github.com//`. Using the [Sourcegraph.com](https://sourcegraph.com) endpoint is free and your index is treated as User-Generated Content (you own it, as covered in our [Sourcegraph.com terms of service](https://about.sourcegraph.com/terms-dotcom#3-proprietary-rights-and-licenses)). If you run into trouble, or a situation arises where you need all of your index expunged, please reach out to us at [support@sourcegraph.com](mailto:support@sourcegraph.com). + +### Proving ownership of a GitHub repository + + + If you're using Sourcegraph.com or have enabled + [`lsifEnforceAuth`](/admin/config/site-config#lsifEnforceAuth), you need to + supply a GitHub token via the `-github-token` flag when uploading. This token + is used to verify repository ownership. + + +To upload to Sourcegraph.com, use the `-github-token` flag: + +```sh +src code-intel upload -github-token=YourGitHubToken -file=index.scip +``` + +The `src` upload command will try to infer the repository and git commit by invoking git commands on your local clone. If git is not installed, is older than version 2.7.0, or you are running on code outside of a git clone, you will need to also specify the `-repo` and `-commit` flags explicitly. diff --git a/docs/code-navigation/how-to/combining-scip-uploads-from-ci-cd-and-auto-indexing.mdx b/docs/code-navigation/how-to/combining-scip-uploads-from-ci-cd-and-auto-indexing.mdx index 070b3a094..0cfbaad1b 100644 --- a/docs/code-navigation/how-to/combining-scip-uploads-from-ci-cd-and-auto-indexing.mdx +++ b/docs/code-navigation/how-to/combining-scip-uploads-from-ci-cd-and-auto-indexing.mdx @@ -2,7 +2,7 @@ Sourcegraph Enterprise instances can serve many profiles of repository size and build complexity, and therefore provides multiple methods precise SCIP index data for your team's repositories. We currently support: -1. Uploading [SCIP data yourself](/code-navigation/how-to/index-other-languages#upload-scip-data) directly from an already-configured build or continuous integration server, as well as +1. Uploading [SCIP data yourself](/code-navigation/how-to/adding-scip-to-workflows) directly from an already-configured build or continuous integration server, as well as 2. [Auto-indexing](/code-navigation/auto-indexing), which schedules index creation within the Sourcegraph instance. There is nothing preventing users from mix-and-matching these methods (we do it ourselves), but we do have some tips for doing so successfully. diff --git a/docs/code-navigation/how-to/index-a-cpp-repository.mdx b/docs/code-navigation/how-to/index-a-cpp-repository.mdx new file mode 100644 index 000000000..2886ed4af --- /dev/null +++ b/docs/code-navigation/how-to/index-a-cpp-repository.mdx @@ -0,0 +1,83 @@ +# Index a C/C++ repository + +

+ This page describes how you can use [scip-clang](https://github.com/sourcegraph/scip-clang) and [src-cli](https://github.com/sourcegraph/src-cli) to enable precise indexing for C/C++ codebases. +

+ + + Binary releases of scip-clang are available for x86_64 Linux (glibc 2.16+) and x86_64 macOS (arm64 macOS is supported via Rosetta). Windows support is being explored. + + +## CI/CD Integrations and Local Indexing + +For integration with other CI/CD systems, or to test precise indexing of an individual repository, the following instructions can be used +to install and run `scip-clang` and `src-cli`. + + + For general CI patterns (sub-projects, custom build environments, and upload frequency), see [Adding precise code navigation to CI/CD workflows](/code-navigation/how-to/adding-scip-to-workflows). + + +### Prerequisites + +scip-clang requires a [JSON compilation database](https://clang.llvm.org/docs/JSONCompilationDatabase.html) (`compile_commands.json`). How to generate one depends on your build system: + +- **CMake**: Pass `-DCMAKE_EXPORT_COMPILE_COMMANDS=ON` when configuring. +- **Bazel**: Use [hedronvision/bazel-compile-commands-extractor](https://github.com/hedronvision/bazel-compile-commands-extractor) or [grailbio/bazel-compilation-database](https://github.com/grailbio/bazel-compilation-database). +- **Meson**: Use the Ninja backend (the default), which generates `compile_commands.json` automatically. +- **Make / other**: Use [Bear](https://github.com/rizsotto/Bear) to record compiler invocations. + +### Setup + +1. Download the latest scip-clang binary from the [releases page](https://github.com/sourcegraph/scip-clang/releases): + +```shell +curl -L https://github.com/sourcegraph/scip-clang/releases/latest/download/scip-clang-x86_64-linux -o /usr/local/bin/scip-clang +chmod +x /usr/local/bin/scip-clang +``` + +- **macOS**: Replace `linux` with `macos` in the URL. + +2. Install the [Sourcegraph CLI](https://github.com/sourcegraph/src-cli): + +```shell + curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src + chmod +x /usr/local/bin/src +``` + +- **macOS**: Replace `linux` with `darwin` in the URL and choose the appropriate architecture: M1/M2 chips - `arm64`, Intel chips - `amd64`. +- **Windows**: Visit [the CLI repo](https://github.com/sourcegraph/src-cli) for further instructions. + +### Indexing + +1. Build your project first so that any generated source files are present on disk. + +2. `cd` into your project's root and run: + +```shell +scip-clang --compdb-path=compile_commands.json +``` + + + scip-clang must be invoked from the project root, not from a subdirectory. The compilation database path is relative to the current working directory. + + + + scip-clang uses approximately 2 MB of temporary disk space per translation unit and about 2 GB of RAM per core. For large codebases, consider testing on a subset first by slicing the compilation database with jq: + +```shell +jq '.[0:10]' compile_commands.json > compile_commands_subset.json +scip-clang --compdb-path=compile_commands_subset.json +``` + + +3. Upload the data to a Sourcegraph instance with: + +```shell +export SRC_ENDPOINT="YOUR_SOURCEGRAPH_URL" +export SRC_ACCESS_TOKEN="YOUR_SOURCEGRAPH_API_TOKEN" + +cd path/to/index.scip +src code-intel upload +``` + +The upload command will provide a URL you can visit to see the upload status. diff --git a/docs/code-navigation/how-to/index-a-csharp-repository.mdx b/docs/code-navigation/how-to/index-a-csharp-repository.mdx new file mode 100644 index 000000000..9f7ddc4ee --- /dev/null +++ b/docs/code-navigation/how-to/index-a-csharp-repository.mdx @@ -0,0 +1,84 @@ +# Index a C# repository + +

+ This page describes how you can use [scip-dotnet](https://github.com/sourcegraph/scip-dotnet) and [src-cli](https://github.com/sourcegraph/src-cli) to enable precise indexing for C# codebases. +

+ + + scip-dotnet supports C# and Visual Basic via `.sln` and `.csproj` build tools. + + +## CI/CD Integrations and Local Indexing + +For integration with CI/CD systems, or to test precise indexing of an individual repository, the following instructions can be used +to install and run `scip-dotnet` and `src-cli`. + + + For general CI patterns (sub-projects, custom build environments, and upload frequency), see [Adding precise code navigation to CI/CD workflows](/code-navigation/how-to/adding-scip-to-workflows). + + +### Setup + +1. Install the [C# SCIP indexer](https://github.com/sourcegraph/scip-dotnet). + +**Using Docker (easiest):** + +```shell +docker pull sourcegraph/scip-dotnet:latest +``` + +**Using the .NET CLI:** + +Requires [.NET 8.0](https://dotnet.microsoft.com/en-us/download). + +```shell +dotnet tool install --global scip-dotnet +``` + +Verify the installation: + +```shell +scip-dotnet --version +``` + +To update an existing installation: + +```shell +dotnet tool update --global scip-dotnet +``` + +2. Install the [Sourcegraph CLI](https://github.com/sourcegraph/src-cli): + +```shell + curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src + chmod +x /usr/local/bin/src +``` + +- **macOS**: Replace `linux` with `darwin` in the URL and choose the appropriate architecture: M1/M2 chips - `arm64`, Intel chips - `amd64`. +- **Windows**: Visit [the CLI repo](https://github.com/sourcegraph/src-cli) for further instructions. + +### Indexing + +1. `cd` into your project's root (where the `.sln` or `.csproj` file lives) and run: + +```shell + scip-dotnet index # generates a file named index.scip +``` + +Or with Docker: + +```shell +docker run -v $(pwd):/app sourcegraph/scip-dotnet:latest scip-dotnet index +``` + +2. Upload the data to a Sourcegraph instance with: + +```shell +export SRC_ENDPOINT="YOUR_SOURCEGRAPH_URL" +export SRC_ACCESS_TOKEN="YOUR_SOURCEGRAPH_API_TOKEN" + +cd path/to/index.scip +src code-intel upload +``` + +The upload command will provide a URL you can visit to see the upload status. diff --git a/docs/code-navigation/how-to/index-a-go-repository.mdx b/docs/code-navigation/how-to/index-a-go-repository.mdx index 69fd5dacf..7f930c2c1 100644 --- a/docs/code-navigation/how-to/index-a-go-repository.mdx +++ b/docs/code-navigation/how-to/index-a-go-repository.mdx @@ -1,91 +1,63 @@ -# Go SCIP indexing +# Index a Go repository

- This page describes how you can automate data indexing in SCIP for Go - codebases or index data manually. + This page describes how you can use [scip-go](https://github.com/sourcegraph/scip-go) and [src-cli](https://github.com/sourcegraph/src-cli) to enable precise indexing for Go codebases.

-## Automated indexing +## GitHub Actions -Sourcegraph provides the Docker images `sourcegraph/scip-go` and `sourcegraph/src-cli` so that you can easily automate indexing in your favorite CI framework. Note that the `scip-go` image bundles `src-cli`, so the second image may not be necessary. +[sourcegraph/scip-go-action](https://github.com/sourcegraph/scip-go-action) can be used to index a Go repository using GitHub Actions. -The following examples show you how to set up automated indexing in a few popular frameworks. You'll need to substitute the indexer and upload commands with what works for your project locally. If you implement automated indexing in a different framework, feel free to edit this page with instructions! - -### GitHub Actions +The action installs the latest versions of [src-cli](https://github.com/sourcegraph/src-cli) and [scip-go](https://github.com/sourcegraph/scip-go) +and relies on setup of a go environment to build your application, prior to running. ```yaml +name: scip-go on: - - push + push: + +permissions: + contents: read jobs: - scip-go: - # this line will prevent forks of this repo from uploading lsif indexes - if: github.repository == '' - runs-on: ubuntu-latest - container: sourcegraph/scip-go:latest - steps: - - uses: actions/checkout@v1 - - name: Generate SCIP data - run: scip-go - - name: Install src CLI - run: | - curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src - chmod +x /usr/local/bin/src - - name: Upload SCIP data - # this will upload to Sourcegraph.com, you may need to substitute a different command. - # by default, we ignore failures to avoid disrupting CI pipelines with non-critical errors. - run: src code-intel upload -github-token=${{ secrets.GITHUB_TOKEN }} -ignore-upload-failure + scip-go: + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup Go + uses: actions/setup-go@v6 + + - name: Sourcegraph Code Intelligence + uses: sourcegraph/scip-go-action@v1 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + scip-go-version: latest + sourcegraph-url: https://sourcegraph.com/ + sourcegraph-token: ${{ secrets.SRC_ACCESS_TOKEN }} + upload: true ``` -### CircleCI +## CI/CD Integrations and Local Indexing -```yaml -version: 2.1 +For integration with other CI/CD systems, or to test precise indexing of an individual repository, the following instructions can be used +to install and run `scip-go` and `src-cli`. -jobs: - scip-go: - docker: - - image: sourcegraph/scip-go:latest - steps: - - checkout - - run: scip-go - - run: | - curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src - chmod +x /usr/local/bin/src; - # this will upload to Sourcegraph.com, you may need to substitute a different command. - # by default, we ignore failures to avoid disrupting CI pipelines with non-critical errors. - - run: src code-intel upload -github-token=<> -ignore-upload-failure - -workflows: - scip-go: - jobs: - - scip-go -``` + + For general CI patterns (sub-projects, custom build environments, and upload frequency), see [Adding precise code navigation to CI/CD workflows](/code-navigation/how-to/adding-scip-to-workflows). + -### Travis CI +### Setup -```yaml -services: - - docker +1. Install the [Go SCIP indexer](https://github.com/sourcegraph/scip-go): -jobs: - include: - - stage: scip-go - # this will upload to Sourcegraph.com, you may need to substitute a different command. - # by default, we ignore failures to avoid disrupting CI pipelines with non-critical errors. - script: - - | - docker run --rm -v $(pwd):/src -w /src sourcegraph/scip-go:latest /bin/sh -c \ - "scip-go; curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src; chmod +x /usr/local/bin/src; src code-intel upload -github-token=$GITHUB_TOKEN -ignore-upload-failure" +```shell +go install github.com/sourcegraph/scip-go/cmd/scip-go@latest ``` -## Manual indexing - -1. Install the [Go SCIP indexer](https://github.com/sourcegraph/scip-go). - -1. Install the [Sourcegraph CLI](https://github.com/sourcegraph/src-cli) with the following command: +2. Install the [Sourcegraph CLI](https://github.com/sourcegraph/src-cli): -```sh +```shell curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src chmod +x /usr/local/bin/src ``` @@ -93,18 +65,27 @@ jobs: - **macOS**: Replace `linux` with `darwin` in the URL and choose the appropriate architecture: M1/M2 chips - `arm64`, Intel chips - `amd64`. - **Windows**: Visit [the CLI repo](https://github.com/sourcegraph/src-cli) for further instructions. +### Indexing + 1. `cd` into your Go project's root (where the `go.mod` file lives, if you have one) and run: -```sh +```shell scip-go # generates a file named index.scip ``` -1. Upload the data to a Sourcegraph instance with: + + If you have multiple go modules, run scip-go at the root of each module and adjust + the src code-intel upload command with the appropriate `-root` flag for each generated index.scip file. + + +2. Upload the data to a Sourcegraph instance with: -```sh +```shell +export SRC_ENDPOINT="YOUR_SOURCEGRAPH_URL" +export SRC_ACCESS_TOKEN="YOUR_SOURCEGRAPH_API_TOKEN" - src -endpoint= code-intel upload # for private instances - src code-intel upload -github-token= # for public instances +cd path/to/index.scip +src code-intel upload ``` -The upload command will provide a URL you can visit to see the upload status. When the upload is complete, you can visit the repo and check out the difference in code navigation quality! +The upload command will provide a URL you can visit to see the upload status. diff --git a/docs/code-navigation/how-to/index-a-jvm-repository.mdx b/docs/code-navigation/how-to/index-a-jvm-repository.mdx new file mode 100644 index 000000000..3224fddcc --- /dev/null +++ b/docs/code-navigation/how-to/index-a-jvm-repository.mdx @@ -0,0 +1,74 @@ +# Index a JVM repository + +

+ This page describes how you can use [scip-java](https://github.com/sourcegraph/scip-java) and [src-cli](https://github.com/sourcegraph/src-cli) to enable precise indexing for JVM codebases. +

+ + + Set the `JVM_VERSION` environment variable to match your project's Java version (e.g., `8`, `11`, `17`, or `21`). + + +## CI/CD Integrations and Local Indexing + +For integration with other CI/CD systems, or to test precise indexing of an individual repository, the following instructions can be used +to install and run `scip-java` and `src-cli`. + + + For general CI patterns (sub-projects, custom build environments, and upload frequency), see [Adding precise code navigation to CI/CD workflows](/code-navigation/how-to/adding-scip-to-workflows). + + +### Setup + +1. Install the [Java SCIP indexer](https://github.com/sourcegraph/scip-java) using [Coursier](https://get-coursier.io): + +```shell +curl -fLo coursier https://git.io/coursier-cli +chmod +x coursier +./coursier bootstrap --standalone -o scip-java com.sourcegraph:scip-java_2.13:latest --main com.sourcegraph.scip_java.ScipJava +``` + +Alternatively, use the Docker image: + +```shell +docker pull sourcegraph/scip-java:latest +``` + +2. Install the [Sourcegraph CLI](https://github.com/sourcegraph/src-cli): + +```shell + curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src + chmod +x /usr/local/bin/src +``` + +- **macOS**: Replace `linux` with `darwin` in the URL and choose the appropriate architecture: M1/M2 chips - `arm64`, Intel chips - `amd64`. +- **Windows**: Visit [the CLI repo](https://github.com/sourcegraph/src-cli) for further instructions. + +### Indexing + +1. `cd` into your project's root (where `pom.xml`, `build.gradle`, or `build.sbt` lives) and run: + +```shell + scip-java index # generates a file named index.scip +``` + +Or with Docker: + +```shell +docker run -v $(pwd):/sources --env JVM_VERSION=17 sourcegraph/scip-java:latest scip-java index +``` + + + `scip-java` automatically detects your build tool (Gradle, Maven, sbt, Mill, or Bazel). To specify one explicitly, use the `--build-tool` flag, e.g., `scip-java index --build-tool=Gradle`. + + +2. Upload the data to a Sourcegraph instance with: + +```shell +export SRC_ENDPOINT="YOUR_SOURCEGRAPH_URL" +export SRC_ACCESS_TOKEN="YOUR_SOURCEGRAPH_API_TOKEN" + +cd path/to/index.scip +src code-intel upload +``` + +The upload command will provide a URL you can visit to see the upload status. diff --git a/docs/code-navigation/how-to/index-a-python-repository.mdx b/docs/code-navigation/how-to/index-a-python-repository.mdx new file mode 100644 index 000000000..4499f340a --- /dev/null +++ b/docs/code-navigation/how-to/index-a-python-repository.mdx @@ -0,0 +1,80 @@ +# Index a Python repository + +

+ This page describes how you can use [scip-python](https://github.com/sourcegraph/scip-python) and [src-cli](https://github.com/sourcegraph/src-cli) to enable precise indexing for Python codebases. +

+ + + scip-python is a Sourcegraph fork of [Pyright](https://github.com/microsoft/pyright) focused on generating SCIP for Python projects. It requires Python 3.10+ and Node v16 or newer. + + +## CI/CD Integrations and Local Indexing + +For integration with CI/CD systems, or to test precise indexing of an individual repository, the following instructions can be used +to install and run `scip-python` and `src-cli`. + + + For general CI patterns (sub-projects, custom build environments, and upload frequency), see [Adding precise code navigation to CI/CD workflows](/code-navigation/how-to/adding-scip-to-workflows). + + +### Setup + +1. Install the [Python SCIP indexer](https://github.com/sourcegraph/scip-python): + +```shell +npm install -g @sourcegraph/scip-python +``` + +2. Install the [Sourcegraph CLI](https://github.com/sourcegraph/src-cli): + +```shell + curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src + chmod +x /usr/local/bin/src +``` + +- **macOS**: Replace `linux` with `darwin` in the URL and choose the appropriate architecture: M1/M2 chips - `arm64`, Intel chips - `amd64`. +- **Windows**: Visit [the CLI repo](https://github.com/sourcegraph/src-cli) for further instructions. + +### Indexing + +1. `cd` into your Python project's root and run: + +```shell +scip-python index . --project-name=$MY_PROJECT +``` + + + Make sure to activate your virtual environment before running scip-python. If you hit out-of-memory errors, set `NODE_OPTIONS="--max-old-space-size=8192"` before running the command. + + +You can optionally pass `--target-only=src/subdir` to index only a specific directory, or `--environment=path/to/env.json` to supply a packages list instead of relying on pip detection. + +2. Upload the data to a Sourcegraph instance with: + +```shell +export SRC_ENDPOINT="YOUR_SOURCEGRAPH_URL" +export SRC_ACCESS_TOKEN="YOUR_SOURCEGRAPH_API_TOKEN" + +cd path/to/index.scip +src code-intel upload +``` + +The upload command will provide a URL you can visit to see the upload status. + +## Auto-indexing + +Sourcegraph can automatically index your Python repository. To configure auto-indexing, use the following configuration: + +```json +{ + "index_jobs": [{ + "indexer": "sourcegraph/scip-python:autoindex", + "local_steps": ["pip install . || true"], + "indexer_args": ["scip-python", "index", ".", "--project-name", "", "--project-version", "_"], + "steps": [], + "outfile": "", + "root": "" + }], + "shared_steps": [] +} +``` diff --git a/docs/code-navigation/how-to/index-a-ruby-repository.mdx b/docs/code-navigation/how-to/index-a-ruby-repository.mdx new file mode 100644 index 000000000..7755e9853 --- /dev/null +++ b/docs/code-navigation/how-to/index-a-ruby-repository.mdx @@ -0,0 +1,81 @@ +# Index a Ruby repository + +

+ This page describes how you can use [scip-ruby](https://github.com/sourcegraph/scip-ruby) and [src-cli](https://github.com/sourcegraph/src-cli) to enable precise indexing for Ruby codebases. +

+ + + scip-ruby is built on top of [Sorbet](https://sorbet.org). Projects with higher Sorbet adoption (more files marked `# typed: true`) will have better code navigation results. Files marked `# typed: false` or without a sigil are indexed on a best-effort basis. + + +## CI/CD Integrations and Local Indexing + +For integration with CI/CD systems, or to test precise indexing of an individual repository, the following instructions can be used +to install and run `scip-ruby` and `src-cli`. + + + For general CI patterns (sub-projects, custom build environments, and upload frequency), see [Adding precise code navigation to CI/CD workflows](/code-navigation/how-to/adding-scip-to-workflows). + + +### Setup + +1. Install the [Ruby SCIP indexer](https://github.com/sourcegraph/scip-ruby) as a gem: + +**Using a Gemfile:** + +```ruby +gem 'scip-ruby', require: false, group: :development +``` + +Then run: + +```shell +bundle install +``` + +Alternatively, download a pre-built binary from the [releases page](https://github.com/sourcegraph/scip-ruby/releases): + +```shell +curl -L "https://github.com/sourcegraph/scip-ruby/releases/latest/download/scip-ruby-$(uname -m)-$(uname -s | tr '[:upper:]' '[:lower:]')" -o scip-ruby +chmod +x scip-ruby +``` + +2. Install the [Sourcegraph CLI](https://github.com/sourcegraph/src-cli): + +```shell + curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src + chmod +x /usr/local/bin/src +``` + +- **macOS**: Replace `linux` with `darwin` in the URL and choose the appropriate architecture: M1/M2 chips - `arm64`, Intel chips - `amd64`. +- **Windows**: Visit [the CLI repo](https://github.com/sourcegraph/src-cli) for further instructions. + +### Indexing + +1. `cd` into your Ruby project's root and run: + +```shell + bundle exec scip-ruby # generates a file named index.scip +``` + +Or with the standalone binary: + +```shell + ./scip-ruby # generates a file named index.scip +``` + + + If your project has a `sorbet/config` file, scip-ruby automatically detects which files to index. Otherwise, pass a path argument: `scip-ruby .` + + +2. Upload the data to a Sourcegraph instance with: + +```shell +export SRC_ENDPOINT="YOUR_SOURCEGRAPH_URL" +export SRC_ACCESS_TOKEN="YOUR_SOURCEGRAPH_API_TOKEN" + +cd path/to/index.scip +src code-intel upload +``` + +The upload command will provide a URL you can visit to see the upload status. diff --git a/docs/code-navigation/how-to/index-a-rust-repository.mdx b/docs/code-navigation/how-to/index-a-rust-repository.mdx new file mode 100644 index 000000000..fb7893012 --- /dev/null +++ b/docs/code-navigation/how-to/index-a-rust-repository.mdx @@ -0,0 +1,68 @@ +# Index a Rust repository + +

+ This page describes how you can use [rust-analyzer](https://rust-analyzer.github.io/) and [src-cli](https://github.com/sourcegraph/src-cli) to enable precise indexing for Rust codebases. +

+ + + Unlike other SCIP indexers, Rust uses rust-analyzer (the Rust language server) which has built-in SCIP generation support. No separate indexer package is needed. + + +## CI/CD Integrations and Local Indexing + +For integration with CI/CD systems, or to test precise indexing of an individual repository, the following instructions can be used +to install and run `rust-analyzer` and `src-cli`. + + + For general CI patterns (sub-projects, custom build environments, and upload frequency), see [Adding precise code navigation to CI/CD workflows](/code-navigation/how-to/adding-scip-to-workflows). + + +### Setup + +1. Install [rust-analyzer](https://rust-analyzer.github.io/) via rustup: + +```shell +rustup component add rust-analyzer +``` + +Alternatively, download a pre-built binary from the [releases page](https://github.com/rust-lang/rust-analyzer/releases). + +2. Install the [Sourcegraph CLI](https://github.com/sourcegraph/src-cli): + +```shell + curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src + chmod +x /usr/local/bin/src +``` + +- **macOS**: Replace `linux` with `darwin` in the URL and choose the appropriate architecture: M1/M2 chips - `arm64`, Intel chips - `amd64`. +- **Windows**: Visit [the CLI repo](https://github.com/sourcegraph/src-cli) for further instructions. + +### Indexing + +1. `cd` into your Rust project's root (where the `Cargo.toml` file lives) and run: + +```shell + rust-analyzer scip . +``` + +This generates a file named `index.scip` by default. You can customize the output path with `--output`: + +```shell + rust-analyzer scip . --output path/to/output.scip +``` + + + The project must be a valid Cargo project with a `Cargo.toml` file. Use `--exclude-vendored-libraries` to skip indexing vendored code. + + +2. Upload the data to a Sourcegraph instance with: + +```shell +export SRC_ENDPOINT="YOUR_SOURCEGRAPH_URL" +export SRC_ACCESS_TOKEN="YOUR_SOURCEGRAPH_API_TOKEN" + +cd path/to/index.scip +src code-intel upload +``` + +The upload command will provide a URL you can visit to see the upload status. diff --git a/docs/code-navigation/how-to/index-a-typescript-and-javascript-repository.mdx b/docs/code-navigation/how-to/index-a-typescript-and-javascript-repository.mdx index a2b3d77f3..308592d9a 100644 --- a/docs/code-navigation/how-to/index-a-typescript-and-javascript-repository.mdx +++ b/docs/code-navigation/how-to/index-a-typescript-and-javascript-repository.mdx @@ -1,133 +1,59 @@ -# Index a TypeScript or JavaScript repository +# TypeScript and JavaScript SCIP indexing

- This page describes how to create an index for JavaScript and TypeScript - projects and uploading it to Sourcegraph. + This page describes how you can use [scip-typescript](https://github.com/sourcegraph/scip-typescript) + and [src-cli](https://github.com/sourcegraph/src-cli) to enable precise indexing for TypeScript and JavaScript codebases.

-We will use [`scip-typescript`](https://github.com/sourcegraph/scip-typescript) to create the index and the [Sourcegraph CLI](https://github.com/sourcegraph/src-cli) to upload it to Sourcegraph. - -## Indexing in CI using scip-typescript directly - -In this approach, you will directly install `scip-typescript` and `src-cli` in CI. This is particularly useful if you are already using some other Docker image for your build. - -Here is an example using GitHub Actions to create and upload an index for a TypeScript project. - -```yaml -jobs: - create-index-and-upload: - # prevent forks of this repo from uploading lsif indexes - if: github.repository == '' - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v3 - - name: Install dependencies - run: npm install - - name: Install scip-typescript - run: npm install -g @sourcegraph/scip-typescript - - name: Generate index - uses: scip-typescript index - - name: Install src-cli - run: | - curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src - chmod +x /usr/local/bin/src - - name: Upload index - run: src code-intel upload -github-token='${{ secrets.GITHUB_TOKEN }}' -no-progress - env: - SRC_ENDPOINT: https://sourcegraph.com/ -``` - - - `src-cli` ignores index upload failures by default to avoid disrupting CI - pipelines with non-critical errors. - - -On CI providers other than GitHub Actions, you may need to explicitly install [Node.js](https://nodejs.org/) as a first step. See the [`scip-typescript` README](https://github.com/sourcegraph/scip-typescript) for the list of supported Node.js versions. - -### Optional scip-typescript flags - -The exact `scip-typescript` invocation will vary based on your configuration. For example: - -- If you are indexing a JavaScript project instead of TypeScript, add the `--infer-tsconfig` flag. - - ```sh - scip-typescript index --infer-tsconfig - ``` - -- If you are indexing a project using Yarn workspaces, add the `--yarn-workspaces` flag. - - ```sh - scip-typescript index --yarn-workspaces - ``` +## CI/CD Integrations and Local Indexing -- If you are indexing a project using Pnpm workspaces, add the `--pnpm-workspaces` flag. +For integration with other CI/CD systems, or to test precise indexing of an individual repository, the following instructions can be used +to install and run `scip-typescript` and `src-cli`. - ```sh - scip-typescript index --pnpm-workspaces - ``` - -## Indexing in CI using the scip-typescript Docker image + + For general CI patterns (sub-projects, custom build environments, and upload frequency), see [Adding precise code navigation to CI/CD workflows](/code-navigation/how-to/adding-scip-to-workflows). + -Sourcegraph provides a Docker image for `sourcegraph/scip-typescript`, which bundles `src-cli` for convenience. +### Setup -Here is an example using the `scip-typescript` Docker image with GitHub Actions to index a TypeScript project. +1. Install the [TypeScript SCIP indexer](https://github.com/sourcegraph/scip-typescript): -```yaml -jobs: - create-and-upload-index: - # prevent forks of this repo from uploading lsif indexes - if: github.repository == '' - runs-on: ubuntu-latest - container: sourcegraph/scip-typescript:latest - steps: - - uses: actions/checkout@v3 - - name: Install dependencies - run: npm install - - name: Generate index - run: scip-typescript index - - name: Upload index - run: src code-intel upload -github-token=${{ secrets.GITHUB_TOKEN }} -no-progress - env: - SRC_ENDPOINT: https://sourcegraph.com/ +```shell +npm install -g @sourcegraph/scip-typescript ``` -If you are indexing a JavaScript codebase or a project using Yarn workspaces, tweak the `scip-typescript` invocation as documented in the [Optional scip-typescript flags](#optional-scip-typescript-flags) section. - -## One-off indexing using scip-typescript locally - -Creating one-off indexes and uploading them is valuable as a proof of concept, but indexes are not kept up to date. - -The steps here are similar to those in the previous GitHub Actions example. - -1. Install `scip-typescript`. +2. Install the [Sourcegraph CLI](https://github.com/sourcegraph/src-cli): -```sh - npm install -g @sourcegraph/scip-typescript +```shell + curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src + chmod +x /usr/local/bin/src ``` -2. Install the Sourcegraph CLI. +- **macOS**: Replace `linux` with `darwin` in the URL and choose the appropriate architecture: M1/M2 chips - `arm64`, Intel chips - `amd64`. +- **Windows**: Visit [the CLI repo](https://github.com/sourcegraph/src-cli) for further instructions. -```sh - curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src - chmod +x /usr/local/bin/src -``` - -The exact invocation may change depending on the OS and architecture. See the [`src-cli` README](https://github.com/sourcegraph/src-cli#installation) for details. +### Indexing -3. `cd` into your project's root (which contains `package.json`/`tsconfig.json`) and run the following: +1. `cd` into your project's root (where `package.json`/`tsconfig.json` lives) and run: -```sh - npm install # Enable (1) type-checking code used from external packages and (2) cross-repo navigation by installing dependencies first with npm or yarn - scip-typescript index # for TypeScript projects +```shell + npm install + scip-typescript index # generates a file named index.scip ``` -If you are indexing a JavaScript codebase or a project using Yarn workspaces, tweak the `scip-typescript` invocation as documented in the [Optional scip-typescript flags](#optional-scip-typescript-flags) section. + + If you are indexing a JavaScript project instead of TypeScript, add the `--infer-tsconfig` flag. + For Yarn workspaces, add `--yarn-workspaces`. For Pnpm workspaces, add `--pnpm-workspaces`. + + +2. Upload the data to a Sourcegraph instance with: -4. Upload the data to a Sourcegraph instance. +```shell +export SRC_ENDPOINT="YOUR_SOURCEGRAPH_URL" +export SRC_ACCESS_TOKEN="YOUR_SOURCEGRAPH_API_TOKEN" -```sh - SRC_ENDPOINT= src code-intel upload # for private instances - src code-intel upload -github-token= # for public instances +cd path/to/index.scip +src code-intel upload ``` -The upload command will provide a URL you can visit to see the upload status. Once the upload has finished processing, you can visit the repo and enjoy precise code navigation! +The upload command will provide a URL you can visit to see the upload status. diff --git a/docs/code-navigation/how-to/index-other-languages.mdx b/docs/code-navigation/how-to/index-other-languages.mdx deleted file mode 100644 index 804ddfb5b..000000000 --- a/docs/code-navigation/how-to/index-other-languages.mdx +++ /dev/null @@ -1,86 +0,0 @@ -# Index other languages - -

- This guide is meant to provide instructions for how to enable precise code - navigation for any programming language. -

- -The general steps for enabling precise code navigation are as follows: - -## Install Sourcegraph CLI - -The [Sourcegraph CLI](/cli/) is used for uploading SCIP data to your Sourcegraph instance (replace `linux` with `darwin` for macOS): - -```sh -curl -L https://sourcegraph.com/.api/src-cli/src_linux_amd64 -o /usr/local/bin/src -chmod +x /usr/local/bin/src -``` - -## Install SCIP indexer - -An SCIP indexer is a command line tool that performs code analysis of source code and outputs a file with metadata containing all the definitions, references, and hover documentation in a project in SCIP. The SCIP file is then uploaded to Sourcegraph to power code navigation features. - -Install the [indexer](/code-navigation/writing-an-indexer) for the required programming language of your repository by following the instructions in the indexer's README - -## Generate SCIP data - -To generate the SCIP data for your repository run the command in the _generate SCIP data_ step found in the README of the installed indexer. - -## Upload SCIP data - -The upload step is the same for all languages. Make sure the current working directory is a path inside your repository, then use the Sourcegraph CLI to upload the SCIP file: - -### To a private Sourcegraph instance (on prem) - -```sh -SRC_ENDPOINT=https://sourcegraph.mycompany.com src code-intel upload -file=index.scip -``` - -### To cloud based Sourcegraph.com - -```sh -src code-intel upload -github-token=YourGitHubToken -file=dump.lsif -``` - -The `src-cli` upload command will try to infer the repository and git commit by invoking git commands on your local clone. If git is not installed, is older than version 2.7.0 or you are running on code outside of a git clone, you will need to also specify the `-repo` and `-commit` flags explicitly. - - - If you're using Sourcegraph.com or have enabled - [`lsifEnforceAuth`](/admin/config/site-config#lsifEnforceAuth) you need to - [supply a GitHub token](#proving-ownership-of-a-github-repository) supplied - via the `-github-token` flag in the command above. - - -Successful output will appear similar to the following example. - -![Uploading a code graph index via the Sourcegraph CLI](https://storage.googleapis.com/sourcegraph-assets/docs/images/code-intelligence/sg-3.34/uploads/src-lsif-upload.gif) - -## Automate code indexing - -Now that you have successfully enabled code navigation for your repository, you can automate source code indexing to ensure precise code navigation stays up to date with the most recent code changes in the repository. See our [continuous integration guide](/code-navigation/how-to/adding-scip-to-workflows) to setup automation. - -## Troubleshooting - -### Testing - - - Make sure you have configured your Sourcegraph instance and [enabled precise - code - navigation](/code-navigation/precise-code-navigation#setting-up-code-navigation-for-your-codebase). - - -Once LSIF data has uploaded, open the Sourcegraph UI or your code host (i.e. GitHub) and navigate to any code file that was part of the repository that was analyzed by the LSIF indexer. Hover over a symbol, variable or function name in the file, you should now see rich LSIF metadata as the source for hover-tooltips, definitions, and references. - -### Error Logs - -To view code graph indexer processing failures go to **Repository settings > Code graph data > Activity for this repository** in your Sourcegraph instance. Failures can occur if the LSIF data is invalid (e.g., malformed indexer output), or problems were encountered during processing (e.g., system-level bug, flaky connections, etc). If you are unable to resolve the issue, please [contact support](https://help.sourcegraph.com/hc/en-us/requests/new). - -### Common Errors - -Possible errors that can happen during upload include: - -- Clone in progress: the instance doesn't have the necessary data to process your upload yet, retry in a few minutes -- Unknown repository (404): check your `-endpoint` and make sure you can view the repository on your Sourcegraph instance -- Invalid commit (404): try visiting the repository at that commit on your Sourcegraph instance to trigger an update -- Invalid auth when using Sourcegraph.com or when [`lsifEnforceAuth`](/admin/config/site-config#lsifEnforceAuth) is `true` (401 for an invalid token or 404 if the repository cannot be found on GitHub.com): make sure your GitHub token is valid and that the repository is correct -- Unexpected errors (500s): reach out to your Sourcegraph support team for assistance diff --git a/docs/code-navigation/how-to/index.mdx b/docs/code-navigation/how-to/index.mdx index d6ba7665e..0757ce3ec 100644 --- a/docs/code-navigation/how-to/index.mdx +++ b/docs/code-navigation/how-to/index.mdx @@ -2,7 +2,12 @@ - [Adding precise code navigation to CI/CD workflows](/code-navigation/how-to/adding-scip-to-workflows) - [Combining SCIP uploads from CI/CD and auto-indexing](/code-navigation/how-to/combining-scip-uploads-from-ci-cd-and-auto-indexing) -- [Go SCIP indexing](/code-navigation/how-to/index-a-go-repository) +- [Index a C/C++ repository](/code-navigation/how-to/index-a-cpp-repository) +- [Index a C# repository](/code-navigation/how-to/index-a-csharp-repository) +- [Index a Go repository](/code-navigation/how-to/index-a-go-repository) +- [Index a JVM repository](/code-navigation/how-to/index-a-jvm-repository) +- [Index a Python repository](/code-navigation/how-to/index-a-python-repository) +- [Index a Ruby Repository](/code-navigation/how-to/index-a-ruby-repository) +- [Index a Rust repository](/code-navigation/how-to/index-a-rust-repository) - [Index a TypeScript or JavaScript repository](/code-navigation/how-to/index-a-typescript-and-javascript-repository) -- [Index other languages](/code-navigation/how-to/index-other-languages) - [Code intelligence policies resource usage best practices](/code-navigation/how-to/policies-resource-usage-best-practices) diff --git a/docs/code-navigation/precise-code-navigation.mdx b/docs/code-navigation/precise-code-navigation.mdx index 5400a4308..78b6725fc 100644 --- a/docs/code-navigation/precise-code-navigation.mdx +++ b/docs/code-navigation/precise-code-navigation.mdx @@ -5,57 +5,46 @@ Available via the Web app. -

Learn and understand about Precise Code Navigation.

+

Learn and understand Precise Code Navigation.

Precise Code Navigation is an opt-in feature that is enabled from your admin settings and requires you to upload indexes for each repository to your Sourcegraph instance. Once setup is complete on Sourcegraph, precise code navigation is available for use across popular development tools: -- On the Sourcegraph web UI -- On code files on your code host, via [integrations](/integration/) -- On diffs in your code review tool, via integrations -- Via the [Sourcegraph API](/api/graphql) +- On the Sourcegraph web UI +- On code files on your code host, via [integrations](/integration/) +- On diffs in your code review tool, via integrations +- Via the [Sourcegraph API](/api/graphql) +- Via the [Sourcegraph MCP Tools](api/mcp): `go_to_definition` and `find_references` Sourcegraph automatically uses Precise Code Navigation whenever available, and Search-based Code Navigation is used as a fallback when precise navigation is not available. Precise code navigation relies on the open source [SCIP Code Intelligence Protocol](https://github.com/sourcegraph/scip), which is a language-agnostic protocol for indexing source code. -## Setting up code navigation for your codebase +## Setting up precise code navigation - - Auto-indexing is a fast way to get started, but for complex or - authenticated builds we recommend [running SCIP indexing in your CI - pipeline](/code-navigation/how-to/adding-scip-to-workflows). This reuses - your existing build configuration and secrets, delivering more reliable, - repeatable precise code navigation. For best results, verify the indexer - runs successfully on your machine first, then add it as a CI job. - - -1. **Manual indexing**. Index a repository and upload it to your Sourcegraph instance: - - - [Index a Go repository](/code-navigation/how-to/index-a-go-repository#manual-indexing) - - [Index a TypeScript or JavaScript repository](/code-navigation/how-to/index-a-typescript-and-javascript-repository#manual-indexing) - - [Index a Java, Scala, or Kotlin repository](https://sourcegraph.github.io/scip-java/docs/getting-started.html) - - [Index a Python repository](https://sourcegraph.com/github.com/sourcegraph/scip-python) - - [Index a Ruby repository](https://sourcegraph.com/github.com/sourcegraph/scip-ruby) +Precise Code Navigation requires language-specific indexes to be generated and uploaded to your Sourcegraph instance. Start with the guide for your preferred language. Each guide covers indexer installation, index generation, uploading, and CI integration steps specific to that language. See the [indexers page](/code-navigation/writing-an-indexer) for a detailed breakdown of each indexer's status. -2. [**Automate indexing via CI**](/code-navigation/how-to/adding-scip-to-workflows): Add indexing and uploading to your CI setup. -3. [**Auto-indexing**](/code-navigation/auto-indexing#enable-auto-indexing): Sourcegraph will automatically index your repositories and enable precise code navigation for them. +| Language | Indexer | Status | Setup guide | +| ---------------------- | --------------------------------------------------------------------------------- | ---------------------- | ---------------------------------------------------------------------------------- | +| Go | [scip-go](https://sourcegraph.com/github.com/sourcegraph/scip-go) | 🟢 Generally available | [Go guide](/code-navigation/how-to/index-a-go-repository) | +| TypeScript, JavaScript | [scip-typescript](https://sourcegraph.com/github.com/sourcegraph/scip-typescript) | 🟢 Generally available | [TS/JS guide](/code-navigation/how-to/index-a-typescript-and-javascript-repository) | +| C, C++, CUDA | [scip-clang](https://sourcegraph.com/github.com/sourcegraph/scip-clang) | 🟢 Generally available | [C/C++ guide](/code-navigation/how-to/index-a-cpp-repository) | +| Java, Kotlin, Scala | [scip-java](https://sourcegraph.com/github.com/sourcegraph/scip-java) | 🟢 Generally available | [JVM guide](/code-navigation/how-to/index-a-jvm-repository) | +| Rust | [rust-analyzer](https://sourcegraph.com/github.com/rust-lang/rust-analyzer) | 🟢 Generally available | [Rust guide](/code-navigation/how-to/index-a-rust-repository) | +| Python | [scip-python](https://sourcegraph.com/github.com/sourcegraph/scip-python) | 🟢 Generally available | [Python guide](/code-navigation/how-to/index-a-python-repository) | +| Ruby | [scip-ruby](https://sourcegraph.com/github.com/sourcegraph/scip-ruby) | 🟢 Generally available | [Ruby guide](/code-navigation/how-to/index-a-ruby-repository) | +| C#, Visual Basic | [scip-dotnet](https://github.com/sourcegraph/scip-dotnet) | 🟢 Generally available | [C# guide](/code-navigation/how-to/index-a-csharp-repository) | -## Supported languages and indexers +Once you've verified the indexer works locally, you can automate it as part of your CI/CD pipelines: -Precise Code Navigation requires language-specific indexes to be generated and uploaded to your Sourcegraph instance. We currently have precise code navigation support for the languages below. See the [indexers page](/code-navigation/writing-an-indexer) for a detailed breakdown of each indexer's status. +- [**CI/CD integration patterns**](/code-navigation/how-to/adding-scip-to-workflows) — general guidance for GitHub Actions, CircleCI, and other CI frameworks (sub-projects, custom build environments, upload frequency). +- [**Auto-indexing**](/code-navigation/auto-indexing#enable-auto-indexing) — let Sourcegraph automatically index your repositories using [executors](/admin/executors/). -| Language | Indexer | Status | -| ---------------------- | --------------------------------------------------------------------------------- | ---------------------- | -| Go | [scip-go](https://sourcegraph.com/github.com/sourcegraph/scip-go) | 🟢 Generally available | -| TypeScript, JavaScript | [scip-typescript](https://sourcegraph.com/github.com/sourcegraph/scip-typescript) | 🟢 Generally available | -| C, C++, CUDA | [scip-clang](https://sourcegraph.com/github.com/sourcegraph/scip-clang) | 🟢 Generally available | -| Java, Kotlin, Scala | [scip-java](https://sourcegraph.com/github.com/sourcegraph/scip-java) | 🟢 Generally available | -| Rust | [rust-analyzer](https://sourcegraph.com/github.com/rust-lang/rust-analyzer) | 🟢 Generally available | -| Python | [scip-python](https://sourcegraph.com/github.com/sourcegraph/scip-python) | 🟢 Generally available | -| Ruby | [scip-ruby](https://sourcegraph.com/github.com/sourcegraph/scip-ruby) | 🟢 Generally available | -| C#, Visual Basic | [scip-dotnet](https://github.com/sourcegraph/scip-dotnet) | 🟢 Generally available | - -The easiest way to configure precise code navigation is with [auto-indexing](/code-navigation/auto-indexing). This feature uses [Sourcegraph executors](/admin/executors/) to automatically create indexes for the code, keeping precise code navigation available and up-to-date. + + For complex or authenticated builds, we recommend [running SCIP indexing + in your CI pipeline](/code-navigation/how-to/adding-scip-to-workflows) + rather than auto-indexing. CI reuses your existing build configuration and + secrets, delivering more reliable, repeatable precise code navigation. + ## Precise navigation examples diff --git a/docs/code-navigation/troubleshooting.mdx b/docs/code-navigation/troubleshooting.mdx index 3fa74214a..e9aaf4849 100644 --- a/docs/code-navigation/troubleshooting.mdx +++ b/docs/code-navigation/troubleshooting.mdx @@ -96,3 +96,29 @@ Hovering over identifiers in the source file should fire off requests to the API ![Network waterfall](https://storage.googleapis.com/sourcegraph-assets/Docs/network-waterfall.png) ![Request headers](https://storage.googleapis.com/sourcegraph-assets/Docs/network-description.png) + +## Upload troubleshooting + +### Verifying precise code navigation + +Once SCIP data has been uploaded, navigate to any code file in the uploaded repository on your Sourcegraph instance. Hover over a symbol, variable, or function name — you should see rich SCIP metadata powering hover tooltips, definitions, and references. + + + Make sure you have configured your Sourcegraph instance and [enabled precise + code + navigation](/code-navigation/precise-code-navigation#setting-up-code-navigation-for-your-codebase). + + +### Error logs + +To view code graph indexer processing failures go to **Repository settings > Code graph data > Activity for this repository** in your Sourcegraph instance. Failures can occur if the SCIP data is invalid (e.g., malformed indexer output), or problems were encountered during processing (e.g., system-level bug, flaky connections, etc). If you are unable to resolve the issue, please [contact support](https://help.sourcegraph.com/hc/en-us/requests/new). + +### Common upload errors + +Possible errors that can happen during upload include: + +- Clone in progress: the instance doesn't have the necessary data to process your upload yet, retry in a few minutes +- Unknown repository (404): check your `-endpoint` and make sure you can view the repository on your Sourcegraph instance +- Invalid commit (404): try visiting the repository at that commit on your Sourcegraph instance to trigger an update +- Invalid auth when using Sourcegraph.com or when [`lsifEnforceAuth`](/admin/config/site-config#lsifEnforceAuth) is `true` (401 for an invalid token or 404 if the repository cannot be found on GitHub.com): make sure your GitHub token is valid and that the repository is correct +- Unexpected errors (500s): reach out to your Sourcegraph support team for assistance diff --git a/docs/code-navigation/writing-an-indexer.mdx b/docs/code-navigation/writing-an-indexer.mdx index b7131e927..cd2b13b6a 100644 --- a/docs/code-navigation/writing-an-indexer.mdx +++ b/docs/code-navigation/writing-an-indexer.mdx @@ -210,17 +210,17 @@ We may lack the language expertise or bandwidth to implement certain features on description="Learn how to create an index for JavaScript and TypeScript projects and uploading it to Sourcegraph." />