Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iszero for Tangents #620

Merged
merged 2 commits into from
May 9, 2023
Merged

iszero for Tangents #620

merged 2 commits into from
May 9, 2023

Conversation

oxinabox
Copy link
Member

@oxinabox oxinabox commented May 8, 2023

closes #619
This makes for a nice canonical way to decide if something should be pulled back.
Since even complicated structures like:

julia> iszero([0.0, NoTangent(), [ZeroTangent(), Tangent{Foo}()]])
true

work right.

So this is better than just checking top-level type.
It is much cheaper to check if something is zero, even if it involved elementwise work than the vast majority of pullbacks so i think a case can be made that this should always been done by the AD. Or at least the AD should have special cases for things to treat as as always nonzero like dense numerical mastrixes <:Array{<:Real} (which are also things that rules support).

This should go well with FluxML/Zygote.jl#1389

src/tangent_types/tangent.jl Outdated Show resolved Hide resolved
test/tangent_types/tangent.jl Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented May 8, 2023

Codecov Report

Patch coverage: 33.33% and project coverage change: -0.21 ⚠️

Comparison is base (79ba4ef) 94.10% compared to head (cc7b906) 93.90%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #620      +/-   ##
==========================================
- Coverage   94.10%   93.90%   -0.21%     
==========================================
  Files          14       14              
  Lines         899      902       +3     
==========================================
+ Hits          846      847       +1     
- Misses         53       55       +2     
Impacted Files Coverage Δ
src/tangent_types/tangent.jl 85.25% <33.33%> (-1.02%) ⬇️

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@oxinabox oxinabox merged commit 7ccf8ed into main May 9, 2023
@oxinabox oxinabox deleted the ox/zeromore branch May 9, 2023 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

overload iszero for Tangent
2 participants