Skip to content

Add multiple code languages - #14

Draft
enaix wants to merge 4 commits into
masterfrom
feat/multilang
Draft

Add multiple code languages#14
enaix wants to merge 4 commits into
masterfrom
feat/multilang

Conversation

@enaix

@enaix enaix commented Jun 17, 2026

Copy link
Copy Markdown
Owner

This PR adds R and Julia support to the grader.

  • Add R install to environments/runtime/Dockerfile and python/Dockerfile
  • Add Julia install to the same dockerfiles
  • Add essential R packages (especially to the manual grader)
  • Add essential Julia packages (especially to the manual grader)
  • Add R code support to the code grader
  • Add Julia code support to the code grader
  • Modify net policies to allow R/Julia packages

Next steps:

  • Add grader-side python isolation (R)
  • Add grader-side python isolation (Julia)
  • Make sure that data conversion is correct (especially for competitions with advanced pandas features)

Agents TODO:

  • Modify ReAct to work with R/Julia
  • Add agent-side python binaries isolation and stricter net policies

Limitations:

  • Manual grader dockerfile doesn't handle packages installed by the agent
  • No R cuda support

@enaix

enaix commented Jun 26, 2026

Copy link
Copy Markdown
Owner Author

Proposed foreignlang python execution prevention method:

  • Allow fork/exec syscalls, since it's needed for libraries to function.
  • During the agent execution stage, ban any access to python/libpython (by removing the binaries and replacing them with honeypots) plus utilize a stricter net policy / scan downloaded files after each script exec / download.
  • During the grader execution, prevent access to the python binary and libpython library, juliacall/pythoncall/pycall/other needs to be handled separately (since libpython is already loaded in memory)

Python scanner

Export check_os_python_access(shell_cmd) function to be used in agent tool calls. Checking the files after each call is not reliable, since a single tool call may immediately download and obfuscate the binary. Alternatively, a better network policy should be used (forbid network access to common python download servers).

Assumption: there is no targeted attack on ml2b to provide obfuscated/encrypted binaries to the model.
Note: honeypot binaries will be used for gathering statistics on unauthorized python call attempts.

@enaix

enaix commented Jul 1, 2026

Copy link
Copy Markdown
Owner Author

We can move julia grading to a separate script, so that it's impossible to access libpython using pythoncall. This will unfortunately add an overhead due to saving to disk + loading the dataset

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant