v0.13.3

Flux v0.13.3

Diff since v0.13.2

Merged pull requests:

• Use var to speed up normalisation (#1973) (@mcabbott)

v0.13.2

Flux v0.13.2

Diff since v0.13.1

Closed issues:

• Inconsistent "Julia ecosystem" docs (#1922)
• sigmoid_fast in GRU? (#1967)

Merged pull requests:

• Unify ecosystem.md (#1923) (@Saransh-cpp)
• Updated path to DiffImages.jl (#1964) (@arcAman07)
• Explain stride≠1 case for SamePad (#1965) (@KronosTheLate)
• fast sigmoid (#1968) (@oysteinsolheim)
• CompatHelper: bump compat for ArrayInterface to 6, (keep existing compat) (#1969) (@github-actions[bot])

v0.13.1

Flux v0.13.1

Diff since v0.13.0

Closed issues:

• Batchnorm on GPU for Float64 values (#1897)
• Tag? (#1924)
• DataLoader causes scalar indexing on GPU in Flux v0.13.0 (regression) (#1935)
• Flux.flip with broadcasting warning (#1936)
• Add a workflow to clean-up gh-pages branch? (#1940)
• DimensionMismatch: All data containers must have the same number of observations. (#1941)
• Type instability in Recur for 3 dimensional arrays (#1947)
• What is the idiomatic way to get training loss from gradient()? (#1950)
• Dropout erroring on latest CUDA (#1960)
• AdaBelief issues (#1962)

Merged pull requests:

• Add a ton of doctests + fix outdated documentation in .md files (#1916) (@Saransh-cpp)
• Get the DocBot up again! (#1937) (@Saransh-cpp)
• Broadcasting replaced with comprehension in the Flux.flip function. (#1938) (@fpartl)
• Fix type instabilities in apply!(optimizer, ...) (#1942) (@ancapdev)
• Add a workflow to delete PR previews (#1943) (@Saransh-cpp)
• Fix for progress logging to non-VS Code loggers (#1944) (@darsnack)
• Add Base.firstindex(c::Chain) = 1 (#1945) (@KronosTheLate)
• Recur type stability for 3d arrays (#1948) (@Marcovela)
• Resolve two warnings in the test suite (#1951) (@mcognetta)
• Update documentation on Split layer (#1953) (@JLDC)
• [docs] suggest using ADAM with LR=1 when combined with ExpDecay (#1955) (@ericphanson)
• Type stable conv_reshape_bias and AD-friendly ConvDims helpers (#1956) (@ToucheSir)
• onehotbatch with CuArray (#1959) (@CarloLucibello)
• AdaBelief bias correction (#1963) (@cossio)

v0.13.0

Flux v0.13.0

Changes in NEWS.md

Diff since v0.12.10

Closed issues:

• DepthwiseConv does not run on GPU (#459)
• Flux type piracy breaks REPL completions (#629)
• Cannot do double iteration of DataLoader (#1227)
• elu activation fails on nested pullbacks on GPU (#1383)
• Training not working for 1D types (#1479)
• adjoint of conv adjoint. (#1665)
• pullback's back returns unexpected size if some parameters are not used (#1601)
• Allow specification of RNG in Dropout (#1617)
• deprecate DepthwiseConv once we have groups in standard conv (#1667)
• Parallel edge-cases (#1685)
• Layer printing interferes with different element types (#1690)
• Normalization Layers not interating well with destructure/restructure (#1727)
• missing docstring for Flux.params and trainable (#1732)
• inconsistency between params and destructure (#1733)
• Parameter Sharing breaks destructure (#1767)
• Remove Juno.jl dependency (#1779)
• Flux.destructure's restructure fails in the gradient if loss does not use all parameters (#1826)
• Flux.chunk for multi-dimensional arrays (#1841)
• onehotbatch performance (#1844)
• Issue taking gradients of Chains on GPU (#1853)
• Chain forgets names under fmap (#1857)
• Recurrent 3d interface uses a lot of memory (#1872)
• Gradient incorrect for Conv-layer and complex numbers (#1876)
• Add Siamese Contrastive Loss function (#1880)
• Urgent GSoC revisions are needed. (#1890)
• Flux v0.12.9 and the Flux.Tracker.gradient is wrong, why? (#1898)
• LoadError UnderVarError: flatten not defined (#1899)
• Proposal: Move params to Zygote (#1900)
• This one is not in use, which one should I use instead in Flux? (#1903)
• ERROR: LoadError: Can't differentiate foreigncall expression (#1904)
• Missing docstring for Flux.Data.Dataloader (#1909)
• Different Julia versions at different places for doctests (#1914)
• Parallel layer behaves diffferently in a Chain than on its own (#1919)
• ADAMW not stable (#1920)
• Chain ignores Base.show function of custom layer (#1929)

Merged pull requests:

• v0.13 deprecations (#1751) (@CarloLucibello)
• Print channel dimensions of Dense like those of Conv (#1658) (@mcabbott)
• Replace unrolled foldl used to evaluate Chain with a better one (#1809) (@mcabbott)
• Zero is a real number (Flux.Nil) (#1830) (@mcabbott)
• Use faster activation functions (#1837) (@mcabbott)
• Add RNG support for Dropout/AlphaDropout (#1849) (@darsnack)
• Fix CI to run on LTS + latest + nightly (#1852) (@darsnack)
• Fix type-stability for normalization layers (#1856) (@pxl-th)
• Use ProgressLogging instead of Juno (#1859) (@darsnack)
• Speed up onehotbatch (#1861) (@mcabbott)
• Simplify trainable, functor and Parallel (#1862) (@mcabbott)
• Replace @adjoint with rrule (#1863) (@mcabbott)
• Depend on Optimisers.jl (#1864) (@mcabbott)
• rationalize CI (#1865) (@CarloLucibello)
• Updated Dropout for more input types. (#1867) (@ShoofLLC)
• fix adamw (#1868) (@CarloLucibello)
• Add OperatorLearning.jl to Flux downstream tests (#1869) (@ChrisRackauckas)
• Mark dropout_mask as non-differentiable (#1870) (@ToucheSir)
• Recurrent benchmarks (#1871) (@mkschleg)
• Changed view to eachslice for folding in recurrent (#1873) (@mkschleg)
• use MLUtils (#1874) (@CarloLucibello)
• Add a structural loadparams! (#1875) (@darsnack)
• Truncated normal initialisation for weights (#1877) (@theabhirath)
• Extending Diagonal (#1881) (@theabhirath)
• rm Flux.Zeros (#1882) (@mcabbott)
• CompatHelper: add new compat entry for SpecialFunctions at version 2, (keep existing compat) (#1883) (@github-actions[bot])
• Make RNN layers accept in => out (#1886) (@mcabbott)
• Speeding up onehotbatch by creating OneHotArray directly (#1888) (@TLipede)
• CompatHelper: bump compat for MLUtils to 0.2, (keep existing compat) (#1889) (@github-actions[bot])
• Addition of Siamese Contrastive Loss function ( Updated ) (#1892) (@arcAman07)
• Buildkite: don't persist registry across runs (#1893) (@ToucheSir)
• Use destructure from Optimisers.jl (#1901) (@mcabbott)
• RFC: Restrict train! to AbstractOptimiser (#1902) (@mcabbott)
• Add dims keywords to some tests (#1906) (@mcabbott)
• Mark initialisations nograd, restrict signatures (#1908) (@mcabbott)
• Add MLUtils's docs and fix some missing docstrings (#1910) (@Saransh-cpp)
• Improvements for LayerNorm (#1911) (@theabhirath)
• Improve docs for initialisation (#1912) (@mcabbott)
• Turn off doctests while building docs (#1915) (@Saransh-cpp)
• dampening -> damping (#1918) (@alhirzel)
• remove DepthwiseConv type in favor of Conv (#1921) (@CarloLucibello)
• Allow activation function for Diagonal (#1925) (@theabhirath)
• Upgrade warnings for v0.13 (#1926) (@mcabbott)
• Rename Diagonal to Scale (#1927) (@mcabbott)
• Fix a code block (#1933) (@prbzrg)

v0.12.10

Flux v0.12.10

Diff since v0.12.9

Closed issues:

• ADAMW not stable (#1920)

Merged pull requests:

• CompatHelper: bump compat for ArrayInterface to 5, (keep existing compat) (#1895) (@github-actions[bot])
• fix adamw (#1868) (@CarloLucibello)

v0.12.9

Flux v0.12.9

Diff since v0.12.8

Closed issues:

• Coverage (#89)
• Support for grouped convolutions (#330)
• onehot.md in docs should not have subtitle "Batches" (#510)
• Repo tagged with the "the-human-brian" (potential typo) (#512)
• RNNs, batching and sequences (#705)
• Model Zoo Housing.jl Example functionality not clear (#769)
• Asymmetric padding fails on gpu models (#775)
• Can't get user defined model to work (#812)
• Cryptic error in Flux#zygote "Can't differentiate foreigncall expression" (#817)
• Passing transposed matrix to softmax causes scalar indexing on GPU, which is very slow (#888)
• Does it support training on multiple GPUs? (#910)
• batched_mul causes a 'cannot take the CPU address of a CuArray' error on GPU (#1090)
• CTC loss (#1135)
• Inconsistent behavior of gradient of empty matrices (#1151)
• Flux.Conv type instability (#1178)
• CUDA.jl (#1194)
• Incorrect types following CUDA.jl refactoring (#1200)
• Got an error, while trying to implement softplus with beta (#1216)
• test regression with recurrent neural networks (#1245)
• regression in RNN with OneHotMatrix and CUDA (#1262)
• Gradient calculation bug re-introduced in Flux v0.10.4 and Zygote v0.4.22 (#1269)
• LSTM "succeeds" on data with incompatible dimensions (#1396)
• Document conv data handling, especially for 1d (#1465)
• Flux.destructure gives DimensionMismatch error in backward pass due to Chain of mutable struct(s) (#1502)
• Adjoints for regularizers? (#1575)
• Zygote error: UndefVarError: S not defined (#1578)
• Warning using Flux on Linux device without CUDA or Nvidia card (#1581)
• Flux downloads CUDA110 Artifacts every time I precompile on Ubuntu (#1600)
• Why does calling the gpu function not return an error when CUDA is unavailable (#1634)
• Flux errors on Julia 1.7 Beta 2 (#1652)
• LLVM 4.x.x compatibility (#1669)
• Add better docs for the LSTM function (#1696)
• Recurrent docs out of sync (#1714)
• Docs haven't built since Aug. 3 (#1723)
• Investigate nightly CI build issues (#1724)
• unsqueeze is not type stable (#1737)
• failing doc tests (#1739)
• Link to "train!" gives 404 page not found error on the website. (#1745)
• Issues model with custom gradient (w.r.t. input variable) layer (#1760)
• Flux.loadparams! is slow. (#1764)
• world age issues when loading a bson file containing a model with flux utility functions (#1769)
• How to fast find source code of function, like Dense() Chain() (#1770)
• How to get the mathematical expression of Neural Network. (#1771)
• How to write a seq of w_i: w_1, w_2, ... , w_1000 (#1773)
• Error when training simple Flux model (#1777)
• Differentiating through my custom struct its restructuring throws an error (#1796)
• Incompatibility with SpecialFunctions 2.0 (#1802)
• Buildkite CI failures with grad test of ConvTranspose + selu (#1804)
• Slowdown when running multiple large models in parallel (#1806)
• ERROR: LoadError: Some tests did not pass: 252 passed, 1 failed, 0 errored, 21 broken. in expression starting at /home/ian/.julia/packages/Flux/BPPNj/test/runtests.jl:11 ERROR: Package Flux errored during testing (#1814)
• Can ExpDecay of learning rate start at some intermediate step? (#1815)
• Optimisers epsilon (#1818)
• Zygote Flux and custom adjoints on GPU (#1828)
• TypeErro in DEQ example: non-boolean (Nothing) used in boolean context #677 (#1846)

Merged pull requests:

• Clarify that params updates (#1752) (@KronosTheLate)
• Add custom model example to docs. (#1758) (@Gregliest)
• Make unsqueeze type stable (#1759) (@cossio)
• Use view for RNN gate slice extraction (#1761) (@ToucheSir)
• Doc update (saving.md): removed outdated info; Typo fix. (#1762) (@NightMachinary)
• Doc update (recurrence.md): fixed incorrect output dimensions, clarified batching. (#1763) (@NightMachinary)
• Expand RNN/LSTM/GRU docs (#1772) (@mcognetta)
• Fix a doctest failure (#1775) (@mcognetta)
• Use conjugates in optimizers to better learn on complex-valued inputs (#1776) (@staticfloat)
• Fix AlphaDropout implementation and add tests (#1781) (@ToucheSir)
• add logo to documentation (#1782) (@kwehmeyer)
• Doc update (training.md): fix DataLoader example in Training section (#1783) (@eliascarv)
• Fix link to train in the docs (#1784) (@logankilpatrick)
• Update train.jl to add a more detailed train! docstring (#1785) (@logankilpatrick)
• Add docstring for params (#1786) (@logankilpatrick)
• Create a PR comment with docs preview link (#1788) (@logankilpatrick)
• Add trilinear Upsample layer (#1792) (@tknopp)
• Tidy up Maxout (#1794) (@mcabbott)
• Simplify mse() to use abs2() (#1795) (@staticfloat)
• Mark destructure gradient test as broken (#1797) (@ToucheSir)
• Fix failing params doctests (#1798) (@ToucheSir)
• Only add PR comment with docs build if the docs label is added (#1799) (@logankilpatrick)
• Add more context on the behavior of the GPU function (#1800) (@logankilpatrick)
• Add warning if the GPU function is called and CUDA is not available (#1801) (@logankilpatrick)
• Add buildkite step to run on Julia LTS (#1805) (@DhairyaLGandhi)
• ExpDecay start step (#1816) (@cossio)
• make eps a parameter of optimisers (#1819) (@cossio)
• Contributor's Guide draft (#1824) (@lilianabs)
• Update conv.jl (#1825) (@rkube)
• CompatHelper: bump compat for ArrayInterface to 4, (keep existing compat) (#1827) (@github-actions[bot])
• Remove "Batches" from one hot section header in docs (#1831) (@darsnack)
• Document disabling GPUs (#1835) (@DhairyaLGandhi)
• Try using latest cu(DNN) binaries (#1836) (@ToucheSir)
• Add news for bump version (#1838) (@DhairyaLGandhi)
• move eps to the end (#1840) (@cossio)
• Add codecov on CI (#1842) (@ToucheSir)
• add token secret for codecov (#1845) (@ToucheSir)
• CompatHelper: bump compat for NNlib to 0.8, NNlibCUDA to 0.2 (keep existing compat) (#1847) (@github-actions[bot])
• Tweak docs about disabling CUDA devices (#1850) (@IanButterworth)

v0.12.8

Flux v0.12.8

Diff since v0.12.7

Closed issues:

• Coverage (#89)
• Flux.train! stops working after the first iteration without an error. (#1692)
• Update Zygote (#1728)
• additional arguments to loss function? (#1730)
• The Purpose and Goals of Flux.jl (#1734)
• FluxML's NumFOCUS Affiliate project application (#1740)
• ConvTranspose does not support groups (#1743)
• deepcopy(nn::Chain) does not deep copy with CuArray weights! (#1747)
• InvalidIRError when putting a model on the GPU (#1754)

Merged pull requests:

• remove Manifest (#1725) (@CarloLucibello)
• add unbatch (#1726) (@CarloLucibello)
• Adds affine and track_stats params to BatchNorm docstring (#1729) (@Mottl)
• add some changes to the beginning of docs (#1736) (@DhairyaLGandhi)
• Fix doc string of Upsample (#1738) (@chunjiw)
• allow groups in ConvTranspose (#1744) (@jw3126)
• Fix Saving and loading model output example (#1746) (@logankilpatrick)
• Fix train! doc string 404 (#1748) (@logankilpatrick)
• Fix @ Functors 404's (#1749) (@logankilpatrick)
• fix CI build (#1750) (@DhairyaLGandhi)

v0.12.7

Flux v0.12.7

Diff since v0.12.6

Closed issues:

• Poor performance relative to PyTorch (#886)
• Recur struct's fields are not type annotated, which is causing run–time dispatch and a significant slowdowns (#1092)
• Bug: lower degree polynomial substitute in gradient chain! (#1188)
• Very slow precompile (>50min) on julia 1.6.0 on Windows (#1554)
• Do not initialize CUDA during precompilation (#1597)
• GRU implementation details (#1671)
• Parallel layer doesn't need to be tied to array input (#1673)
• update! a scalar parameter (#1677)
• Support NamedTuples for Container Layers (#1680)
• Freezing layer parameters still computes all gradients (#1688)
• A demo is 1.5x faster in Flux than tensorflow, both use cpu; while 3.0x slower during using CUDA (#1694)
• Problems with a mixed CPU/GPU model (#1695)
• Flux tests with master fail with signal 11 (#1697)
• [Q] How does Flux.jl work on Apple Silicon (M1)? (#1701)
• Typos in documents (#1706)
• Fresh install of Flux giving errors in precompile (#1710)
• Flux.gradient returns dict of params and nothing (#1713)
• Weight matrix not updating with a user defined initial weight matrix (#1717)
• [Documentation] No logsumexp in NNlib page (#1718)
• Flattened data vs Flux.flatten layer in MNIST MLP in the model zoo (#1722)

Merged pull requests:

• Add WIP docstrings to CPU and GPU (#1632) (@logankilpatrick)
• Add section on Checking GPU Availability (#1633) (@logankilpatrick)
• fix README (#1668) (@DhairyaLGandhi)
• Generalise Parallel forwards pass (#1674) (@DhairyaLGandhi)
• Adding GRUv3 support. (#1675) (@mkschleg)
• Support NamedTuples for Chain + Parallel (#1681) (@mcabbott)
• Adding support for folding RNNs over 3d arrays (#1686) (@mkschleg)
• Update nnlib.md (#1689) (@CarloLucibello)
• fix typo (#1691) (@foldfelis)
• Typo fix (#1693) (@lukemerrick)
• Remove out of date dead code in Conv layers (#1702) (@ToucheSir)
• Gradient definitions for cpu & gpu (#1704) (@mcabbott)
• Fix #1706 (#1707) (@rongcuid)
• Add GPU Adaptor (#1708) (@DhairyaLGandhi)
• Initialize CUDA lazily. (#1711) (@maleadt)
• Update community.md to reflect help wanted != good first issue (#1712) (@logankilpatrick)
• Fix link in README (#1716) (@nilsmartel)
• Add logsumexp to docs (#1719) (@DhairyaLGandhi)

v0.12.6

Flux v0.12.6

Diff since v0.12.5

Merged pull requests:

• Add grouped convolution (#1531) (@DhairyaLGandhi)
• fix deprecations of zeros (#1670) (@DhairyaLGandhi)
• Add GPU activation tests for grouped conv (#1672) (@DhairyaLGandhi)

v0.12.5

Flux v0.12.5

Diff since v0.12.4

Closed issues:

• Hessian vector products (#129)
• Stopping criteria (#227)
• Flux + Julia ecosystem docs (#251)
• RNN unbroadcast on GPU not working (#421)
• Shouldn't gradcheck compares Jacobian? (#462)
• Transition examples in docs to doctests (#561)
• Batch-axis thread parallelism (#568)
• Add tests of ExpDecay (#684)
• Sudden memory leak when training on GPU over many epochs (#736)
• performance variance between macOS / Linux ? (#749)
• onehot ambiguous method (#777)
• Killed while training the model (#779)
• type Method has no field sparam_syms, while @save model (#783)
• Flux#zygote Error in phenomes... Mutating arrays is not supported (#819)
• Custom serialization pass for intermediate states (#845)
• OneHotMatrix does not support map (#958)
• CuArrays + huber_loss iterate(::nothing) error (#1128)
• Can't get Flux (v0.10.3) working for Custom Loss function (#1153)
• Custom loss function on subset of parameters fails (#1371)
• Minimizing sum fails (#1510)
• gpu behaves differently from cu on a Char array (#1517)
• Warn different size inputs in loss functions (#1522)
• Recurrent docs need to be update for v0.12 (#1564)
• Computation of higher order derivatives for recurrent models results in strange errors (#1593)
• Why does DataLoader not throw an error when fed with a 1D vector for the target? (#1599)
• a small error in the documentation... (#1609)
• Slow unnecessary GPU copy of output of gpu(::OffsetArray) (#1610)
• "using Flux" makes type inference fail when there is a Ref{} (#1611)
• @epochs is missing a bracket (#1615)
• Flux Overview Documentation Out of Date (#1621)
• missing kernel for Base.unique (#1622)
• Compilation error on PPC (#1623)
• _restructure as part of the public API? (#1624)
• ERROR: setindex! not defined for Zygote.OneElement{...} (#1626)
• MethodError: Cannot convert an object of type Params to an object of type Float64 (#1629)
• MethodError: no method matching flatten(::Array{Float32,4}) (#1630)
• Where are the cpu() and gpu() functions? (#1631)
• bug in RNN docs (#1638)
• Bug in the current overview documentation (#1642)
• How to tell Flux.jl not to use the GPU? (#1644)
• Missing docs for @functor (#1653)
• typo in the docs/overview section right at the beginning (#1663)

Merged pull requests:

• multiplication of {Transpose, Adjoint} of Array and OneHotVector (#1424) (@gxyd)
• show(::Chain) (#1467) (@mcabbott)
• Add test for show(io, ::OneHotArray) on GPU (#1550) (@darsnack)
• document Join and Split error (#1607) (@magicly)
• fix typo in models overview document (#1608) (@teamclouday)
• fix AdamW and improve decays docs (#1612) (@CarloLucibello)
• use ArrayInterface.restructure in update! (#1613) (@CarloLucibello)
• Warn on reconstruct length mismatch (#1616) (@ToucheSir)
• Forward map(f, ::OneHotLike) to broadcast (#1619) (@darsnack)
• Properly move isbits and numeric arrays to GPU (#1620) (@ToucheSir)
• Update "Composing Optimisers" docs (#1628) (@StevenWhitaker)
• Fixup Dataloader's docstring (#1635) (@mcabbott)
• Add warnings for mismatched sizes in losses (#1636) (@mcabbott)
• updated recurrence.md which fixes #1564 (#1637) (@aditkumar72)
• fix recurrence docs (#1639) (@CarloLucibello)
• Update docstring for Conv to clarify feature dimensions (#1646) (@vivekkumar7089)
• Use correct eltype and rtol in CrossCor tests (#1650) (@ToucheSir)
• add Functors docs (#1654) (@DhairyaLGandhi)
• remove Manifest (#1657) (@CarloLucibello)
• Printing & docstrings for onehot / onehotbatch (#1660) (@mcabbott)
• Deprecate Flux.zeros (#1661) (@mcabbott)