Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Got error while analyzing a Go db #4815

Closed
adilrookout opened this issue Dec 13, 2020 · 18 comments
Closed

Got error while analyzing a Go db #4815

adilrookout opened this issue Dec 13, 2020 · 18 comments
Labels
question Stale

Comments

@adilrookout
Copy link

@adilrookout adilrookout commented Dec 13, 2020

Description of the issue
Created a go database via codeql database create command

Then ran codeql database analyze and got the following error :
Unknown kind "Table" when converting to analysis model.(UNSUPPORTED_KIND).

@adilrookout adilrookout added the question label Dec 13, 2020
@aibaars
Copy link
Contributor

@aibaars aibaars commented Dec 13, 2020

I think the annotation @kind table should be with lower case t .

@adilrookout
Copy link
Author

@adilrookout adilrookout commented Dec 13, 2020

where should i change it ?

@adilrookout
Copy link
Author

@adilrookout adilrookout commented Dec 13, 2020

Here is the problematic query :

/**
 * @name Frequency counts for external APIs that are used with untrusted data
 * @description This reports the external APIs that are used with untrusted data, along with how
 *              frequently the API is called, and how many unique sources of untrusted data flow
 *              to it.
 * @id go/count-untrusted-data-external-api
 * @kind table
 * @tags security external/cwe/cwe-20
 */

import go
import semmle.go.security.ExternalAPIs

from ExternalAPIUsedWithUntrustedData externalAPI
select externalAPI, count(externalAPI.getUntrustedDataNode()) as numberOfUses,
  externalAPI.getNumberOfUntrustedSources() as numberOfUntrustedSources order by
    numberOfUntrustedSources desc

@adilrookout
Copy link
Author

@adilrookout adilrookout commented Dec 13, 2020

@smowton
Copy link
Contributor

@smowton smowton commented Dec 13, 2020

Looks like that really is a bug: obviously the intent is for lgtm.com to display the results in tabular form not as inline alerts, but https://help.semmle.com/QL/learn-ql/writing-queries/query-metadata.html says @kind must be either problem or path-problem.

@smowton
Copy link
Contributor

@smowton smowton commented Dec 13, 2020

With that being said, that kind does seem to be used quite a lot, mostly but not exclusively in tests (e.g. https://github.com/github/codeql/blob/8d4f7e2db7ec3b7559a3e358fd1b8a7203ce7474/cpp/ql/src/Architecture/General%20Class-Level%20Information/HubClasses.ql) -- anyone know why that @kind isn't documented, and where it is or isn't supported?

@adilrookout, what query-set were you running that included this one? It shouldn't be part of any standard query set; what was your full analyze command?

@adilrookout
Copy link
Author

@adilrookout adilrookout commented Dec 15, 2020

codeql database analyze <db path> codeql-go/ql/src/Security --format=csv --output=results.csv

@smowton
Copy link
Contributor

@smowton smowton commented Dec 15, 2020

I recommend running https://github.com/github/codeql-go/blob/main/ql/src/codeql-suites/go-code-scanning.qls instead, which references https://github.com/github/codeql/blob/main/misc/suite-helpers/code-scanning-selectors.yml, which in turn will pick security queries of appropriate types / warning severities. You can copy and customise either one of those files to vary the query selection.

@adilrookout
Copy link
Author

@adilrookout adilrookout commented Dec 15, 2020

Tried and i get the following error :
A fatal error occurred: The QL pack 'codeql-suite-helpers' which is referenced from /Users/adil/work/codeql-home/codeql-go/ql/src/codeql-suites/go-code-scanning.qls cannot be found.

@aibaars
Copy link
Contributor

@aibaars aibaars commented Dec 15, 2020

The suite helpers can be found in https://github.com/github/codeql/tree/main/misc/suite-helpers . If I am not mistaken things should work if you git clone https://github.com/github/codeql as a sibling folder of your codeql-go folder :

git clone https://github.com/github/codeql /Users/adil/work/codeql-home/codeql

@jalseth
Copy link

@jalseth jalseth commented Feb 18, 2021

@aibaars I'm running into the same issue, and that doesn't seem to work. I have both codeql and codeql-go checked out in my $HOME/Code directory and am running into the following error.

$ codeql database analyze my-db $HOME/Code/codeql-go/ql/src/codeql-suites/go-code-scanning.qls --format=csv --output=test01.csv
Running queries.
A fatal error occurred: The QL pack 'codeql-suite-helpers' which is referenced from /Users/james/Code/codeql-go/ql/src/codeql-suites/go-code-scanning.qls cannot be found.

I've tried patching the from: field in the YAML using a relative path, but it results in the same error

$ codeql database analyze my-db $HOME/Code/codeql-go/ql/src/codeql-suites/go-security-extended.qls --format=CSV --output=test01.csv
Running queries.
A fatal error occurred: The QL pack '../../../../codeql/misc/suite-helpers' which is referenced from /Users/james/Code/codeql-go/ql/src/codeql-suites/go-security-extended.qls cannot be found.

@aibaars
Copy link
Contributor

@aibaars aibaars commented Feb 18, 2021

@aibaars I'm running into the same issue, and that doesn't seem to work. I have both codeql and codeql-go checked out in my $HOME/Code directory and am running into the following error.

@jalseth
Did you also unzip the CodeQL CLI in the $HOME/Code folder? If not you should either add a flag --search-path $HOME/Code/codeqlto thecodeql database analyzecommand, or unzip the CodeQL CLI into $HOME/Code`. I normally have the following folder structure:

@jalseth
Copy link

@jalseth jalseth commented Feb 18, 2021

@aibaars Thanks! That was it.

That said, it's quite abnormal for the relative paths to be based on the tool's location, since it can be placed anywhere on disk as long as it's in $PATH. I would expect the relative search path to be based on the current directory, or relative to the configuration files I point it at.

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Apr 16, 2021

This issue is stale because it has been open 14 days with no activity. Comment or remove the stale label in order to avoid having this issue closed in 7 days.

@xanderdunn
Copy link

@xanderdunn xanderdunn commented Feb 7, 2022

I am seeing the same error:

$ ./codeql/codeql database analyze my-proj-database --format=csv --output=./my-proj-output.csv
Running queries.
ERROR: Referenced pack 'codeql/suite-helpers' not found. (/home/xander/codeql-home/codeql-go/ql/src/qlpack.yml:1,1-1)
A fatal error occurred: Could not resolve library path for /home/xander/codeql-home/codeql-go/ql/src

I have the three sibling directories as described above:
Screen Shot 2022-02-06 at 19 35 23

When I run ./codeql/codeql resolve qlpacks it does find the suite-helpers:

codeql/suite-helpers (/home/xander/codeql-home/ql/misc/suite-helpers)

Specifying the --search-path doesn't help either:

$ ./codeql/codeql database analyze my-proj-database --format=csv --output=./my-proj-output.csv --search-path=/home/xander/codeql-home/ql
Running queries.
ERROR: Referenced pack 'codeql/suite-helpers' not found. (/home/xander/codeql-home/codeql-go/ql/src/qlpack.yml:1,1-1)
A fatal error occurred: Could not resolve library path for /home/xander/codeql-home/codeql-go/ql/src

@smowton
Copy link
Contributor

@smowton smowton commented Feb 7, 2022

@xanderdunn have you run install-deps.sh per https://github.com/github/codeql-go#installation ?

@xanderdunn
Copy link

@xanderdunn xanderdunn commented Feb 7, 2022

@smowton That did it, thank you! I was following this guide, which didn't mention it

@smowton
Copy link
Contributor

@smowton smowton commented Feb 7, 2022

Fixed in #7879

@smowton smowton closed this as completed Feb 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Stale
Projects
None yet
Development

No branches or pull requests

5 participants