Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

including externs definitions in "codeql test" #28

Open
franktip opened this issue Apr 30, 2020 · 5 comments
Open

including externs definitions in "codeql test" #28

franktip opened this issue Apr 30, 2020 · 5 comments
Assignees
Labels
CLI

Comments

@franktip
Copy link

@franktip franktip commented Apr 30, 2020

Would it be possible to give the “codeql test” command an option to include externs and/or make this something that one can specify in a config file? Right now, the approach we have taken is to copy/symlink externs definitions into the directory containing test files, but this solution seems unsatisfactory/fragile, particularly if one wants to use the standard externs definitions.

My question relates to JavaScript, but the same issue presumably arises for other languages.

@hmakholm
Copy link
Contributor

@hmakholm hmakholm commented May 1, 2020

Currently we have very limited means for passing through options from the codeql command line to the language-specific parts of database creation. There's a design in the works to remedy that in a structured way, which will hopefully land over the summer.

For the specific point about externs, perhaps someone in @github/codeql-javascript can comment?

@p0
Copy link
Collaborator

@p0 p0 commented May 6, 2020

I briefly discussed this with @github/codeql-javascript. It seems like we would like to assume the externs are always available in the database, and so I think we should make a pre-finalise script that extracts them.

@p0 p0 assigned asgerf May 6, 2020
@asgerf
Copy link

@asgerf asgerf commented Jun 2, 2020

Sorry for taking this long getting back to you @franktip. I'm afraid your workaround will have to do.

For a while we were considering extracting externs in tests by default, but have decided against it for two reasons:

  • Tests will take longer. Even a few seconds/test is noticable.
  • Tests will find results inside the externs, which is surprising, and many of our tests would need to be updated to exclude such results.
@asgerf asgerf closed this Jun 2, 2020
@franktip
Copy link
Author

@franktip franktip commented Jun 2, 2020

Hi Asger,

I understand that you don't want to include the externs by default for performance reasons, but would it be possible to make it an option that users need to specify/enable explicitly in a config file? I would really appreciate it if you could come up with a solution so that I won't have to copy/symlink the externs definitions in my tests.

Let me know what you think.

best regards,

-Frank

@asgerf
Copy link

@asgerf asgerf commented Jun 3, 2020

Right, adding an option for it would indeed be very useful. I'll re-open the issue, but resolution will unfortunately have to wait until the method for passing options has landed (see @hmakholm's comment above).

Currently we have very limited means for passing through options from the codeql command line to the language-specific parts of database creation. There's a design in the works to remedy that in a structured way, which will hopefully land over the summer.

@asgerf asgerf reopened this Jun 3, 2020
@adityasharad adityasharad added the CLI label Jul 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.