Add Codecov step to CI

[brianschubert]

c.f. #520 (comment)

Curious if this will "just work." This is based on the Codecov setup used in the bedevere repo.

[codecov]

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

[brianschubert]

I think the fact that a comment was posted means that the Codecov app is already enabled for this repo and that the organization access token works?

cc @AlexWaygood @JelleZijlstra @Daraan, thoughts on using this in some form?

This would give us a pretty dashboard for monitoring code coverage (example from the bedevere repo: https://app.codecov.io/gh/python/bedevere/). Optionally, we could also enable PR comments (example) and status checks, but this is usable without those as well

[AlexWaygood]

I feel like I have a vague memory of codecov being flaky/unreliable on some previous projects...? But that was a while ago; it might have improved since.

Anyway, I agree with @srittau's comments in #669 (comment) that the current action we're using seems not-great for several reasons. Using an action that other Python-org repos are using means that many more maintainers across the org will be invested in making sure that it doesn't have any security issues. And the fact that it's a GitHub app means that it'll be able to post comments on contributor PRs without the security issues associated with a workflow_call workflow trigger. It's also less GitHub-workflow code for us to maintain, which is always good!

So TL;DR I'm happy to try this out!

[Daraan]

I think the fact that a comment was posted means that the Codecov app is already enabled for this repo and that the organization access token works?

Great it works, I first benched the idea because of the token needed. I guess it works for you as you have organization access :) (That's new right? Congratulations!)
I vaguely remember something that I later read that it might work with public repos but didn't have the time to check up on it yet.

That the dashboard is ready without extra work is a big plus.

[brianschubert]

Cool! I think this PR is in merge-able shape if we want to give this a try. This will enable coverage reporting to the dashboard as well as PR comments. This doesn't enable CI status checks (which are disabled by default by the Python org global YAML), so for now the CI won't fail if coverage drops. That can be configured in the future if desired.

The docs seem to indicate that Codecov always uses the settings files from the current PR branch, so it should be easy to test different settings in followup PRs: https://docs.codecov.com/docs/codecov-yaml#locking-codecov-yaml-to-a-branch

[brianschubert]

Oh, and apparently a coverage report for this PR is already available on the dashboard! https://app.codecov.io/gh/python/typing_extensions/pull/674/tree/src?dropdown=coverage

[AlexWaygood]

LGTM but I'll wait for a second approval from @JelleZijlstra or @srittau before landing