Skip to content

Container in Tekton Hub api pod hits process limit over time due to process leak (sh: can't fork) #2505

@sambou90

Description

@sambou90

We are running a custom Tekton Hub instance on OpenShift. After the api pod has been running for a while, it becomes unusable: running any command inside the container (e.g., ps, ls, or even spawning a shell) fails with the following error:

sh: can't fork: Resource temporarily unavailable

This strongly suggests a process leak inside the Tekton Hub api pod. These processes are likely being spawned but not properly reaped or terminated. In this state the Tekton Hub api pod can not fetch current tekton catalog git content.

Expected Behavior

The Tekton Hub api container should not leak processes over time. The number of active processes should remain stable, and basic shell operations should always work.

Actual Behavior

After the Tekton Hub pod has been running for an extended period of time, it becomes unresponsive due to an internal process leak. It looks like git process are spawned and not cleaned up:

/app $ ps -e
PID USER TIME COMMAND
1 10011100 0:00 /app/api-server
18 10011100 0:00 [git]
25 10011100 0:00 [git]
32 10011100 0:00 [git]
39 10011100 0:00 [git]
46 10011100 0:00 [git]
53 10011100 0:00 [git]
60 10011100 0:00 [git]
67 10011100 0:00 [git]
74 10011100 0:00 [git]
81 10011100 0:00 [git]
88 10011100 0:00 [git]
95 10011100 0:00 [git]
102 10011100 0:00 [git]
105 10011100 0:00 sh
111 10011100 0:00 ps -e

Steps to Reproduce the Problem

  1. Deploy Tekton Hub on OpenShift (or Kubernetes).
  2. Leave the pod running under typical load or idle state.
  3. After a period of time (e.g., hours to days), exec into the pod: oc exec -it <tekton-hub-pod> -- sh
  4. Attempt to run a command: /app $ ps -> sh: can't fork: Resource temporarily unavailable

Additional Info

  • UI Screenshots

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions