Use pod local tmp based builddir instead of shared storage git-repos

[antbob]

The rationale behind this change is to avoid performance, resource consumption and concurrency issues associated with the builddir located on a shared storage like netapp-nfs or aws-efs. The problem with shared storage in this case is builddir often contains an extremely large amount of small files those file systems are ill designed to deal with.

In practical terms after switching to aws-efs to be able to scale the deployment (we do not have netapp-nfs available in our cluster) we are constantly running into run_package_prep/git_prepare_package_sources timing out with:

Operation timed out after 300 s / 600 s

and generally taking a long time (minutes instead of seconds) when working with builddir on shared storage.

AFAIK there is no need to place the builddir on a shared storage, instead the agent local pod storage can be used for this purpose which is much faster and takes less resources. The largest source bases sit at about 4Gb magnitude eg Firefox/Thunderbird and thus the local pod storage has enough space to host the builddir without the need to provision any additional storage resources. As long as cleanup of no longer needed builddirs works fine there should not be any space problems.

The choice to host it under tmp is because there is no need to provision ephemeral storage via deployment files as tmp acts the same way as ephemeral storage in this case anyway.

[nforro]

IIUC this only works because no privileged tools are being run on unpacked sources. That makes sense, but I'm afraid that won't be true forever, see e.g. #814.

[majamassarini]

IIUC this only works because no privileged tools are being run on unpacked sources. That makes sense, but I'm afraid that won't be true forever, see e.g. #814.

I believe #814 is compatible with this change. For now I think we are good to go with it.

[antbob]

missed a spot in cve_applicability_agent. this has been tested in our deployment so far with no issues.

[nforro]

There are a few minor (?) concerns (raised by GPT-5.6 Luna):

• Triage cleanup gap: /tmp builddirs are not cleaned if the triage workflow
  raises before reaching comment_in_jira; large trees can remain until pod
  restart.
• Limited test coverage: Tests verify option/path reuse, but not end-to-end
  RPM cleanup or the new absolute source paths.
• Reduced isolation: Reusing one builddir is probably safe for standard
  sequential RPM specs, but less robust for unusual %prep scripts or
  unexpected concurrent tool calls.

[antbob]

There are a few minor (?) concerns (raised by GPT-5.6 Luna):

• Triage cleanup gap: /tmp builddirs are not cleaned if the triage workflow
  raises before reaching comment_in_jira; large trees can remain until pod
  restart.

This is real and should be addressed in the latest rev along with other similar gaps.

• Limited test coverage: Tests verify option/path reuse, but not end-to-end
  RPM cleanup or the new absolute source paths.

There isnt much i can do about that short of mocking rpmbuilds in unit tests.

• Reduced isolation: Reusing one builddir is probably safe for standard
  sequential RPM specs, but less robust for unusual %prep scripts or
  unexpected concurrent tool calls.

This is again real but a bit nuanced. The only way this can happen is if the LLM response contains (due to hallucination etc) more than 1 call for the same tool eg prep_sources. This obviously should not happen but there is no hard guarantee. There are 2 possible solutions: lock each tool to ensure the same tool isnt called in parallel or set the allow_parallel_tool_calls to false in the chat model. Given how unlikely this is in practice and low cost of possible fallout from it happening i would tend to let this slide, it just doesnt seem to worth the stretch to pay the cost of handling this.

Also i ran this latest rev in our deployment for a little while and even managed to verify the parallel workers case where builddirs got successfully cleaned up. In addition i ran Claude code review asking it to concentrate specifically on builddir leaks and conflicts, it spun 11 agents and spent alot of time and tokens. Not that its any indication of a quality review but for what its worth.

[nforro]

LGTM