Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support non-standard compilation processes #12

Open
mvanotti opened this issue Jan 24, 2020 · 4 comments
Open

Support non-standard compilation processes #12

mvanotti opened this issue Jan 24, 2020 · 4 comments
Labels

Comments

@mvanotti
Copy link

@mvanotti mvanotti commented Jan 24, 2020

In a recent talk with @adityasharad , he mentioned that CodeQL would try to understand when a compiler is being invoked. Some projects use goma to speed up the build process, reusing previously built artifacts.

CodeQL seems to ignore all the artifacts that are obtained via goma.

@p0
Copy link
Collaborator

@p0 p0 commented Jan 27, 2020

To process C++ code, we indeed need to know how to compile it. The easiest way of setting that up is codeql's default, but that indeed relies on observing compilations locally, and distributed build systems like goma or bazel will not work with that approach without disabling the "distributed" aspect.

One possible approach is to leverage compile-commands.json files, as generated by many build systems. The information in them is sufficient to drive the CodeQL tooling, but obtaining such a file is a non-standard build-system-specific and sometimes project-specific process. One possibility would be to add support in CodeQL for creating a database based on such compiler settings files. [It's worth noting that this would not be equivalent to tracing a full local build, since CodeQL takes advantage of information from linker invocations too, and those are not represented in compilation command databases.]

@mvanotti
Copy link
Author

@mvanotti mvanotti commented Jan 27, 2020

Maybe it would be good to have a way of determining what is needed for CodeQL to work properly, and then each build system could figure out how to export that data somehow. That way, having extractors for different build systems depends on the community.

@Manouchehri
Copy link

@Manouchehri Manouchehri commented Mar 18, 2020

Using bear + #9 would be a decent solution in my opinion. =)

@p0
Copy link
Collaborator

@p0 p0 commented Mar 18, 2020

I would expect CodeQL's built-in support to be able to handle any situation where bear would also work. The problem with goma or bazel is that the compilations end up being done on a server process or different machine, and are invisibile to whatever local monitoring you are trying to do of the build process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.