Skip to content

Commit

Permalink
Merge branch 'master' into fg/diskarrays_04
Browse files Browse the repository at this point in the history
  • Loading branch information
meggart authored Oct 21, 2024
2 parents 332c846 + 5bc9146 commit 12b35ad
Show file tree
Hide file tree
Showing 25 changed files with 556 additions and 160 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/ci-binary-builder.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ jobs:
arch:
- x64
steps:
- uses: actions/checkout@v2
- uses: julia-actions/setup-julia@v1
- uses: actions/checkout@v4
- uses: julia-actions/setup-julia@v2
with:
version: ${{ matrix.version }}
arch: ${{ matrix.arch }}
Expand Down
11 changes: 6 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ jobs:
arch:
- x64
steps:
- uses: actions/checkout@v2
- uses: julia-actions/setup-julia@v1
- uses: actions/checkout@v4
- uses: julia-actions/setup-julia@v2
with:
version: ${{ matrix.version }}
arch: ${{ matrix.arch }}
Expand All @@ -43,15 +43,16 @@ jobs:
- uses: julia-actions/julia-runtest@latest
continue-on-error: ${{ matrix.version == 'nightly' }}
- uses: julia-actions/julia-processcoverage@v1
- uses: codecov/codecov-action@v1
- uses: codecov/codecov-action@v4
with:
file: lcov.info
token: ${{ secrets.CODECOV_TOKEN }}
docs:
name: Documentation
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: julia-actions/setup-julia@v1
- uses: actions/checkout@v4
- uses: julia-actions/setup-julia@v2
with:
version: '1'
- run: |
Expand Down
28 changes: 28 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
cff-version: "1.2.0"
authors:
- family-names: Barth
given-names: Alexander
orcid: "https://orcid.org/0000-0003-2952-5997"
doi: 10.5281/zenodo.11067062
message: If you use this software, please cite our article in the
Journal of Open Source Software.
preferred-citation:
authors:
- family-names: Barth
given-names: Alexander
orcid: "https://orcid.org/0000-0003-2952-5997"
date-published: 2024-05-29
doi: 10.21105/joss.06504
issn: 2475-9066
issue: 97
journal: Journal of Open Source Software
publisher:
name: Open Journals
start: 6504
title: "NCDatasets.jl: a Julia package for manipulating netCDF data
sets"
type: article
url: "https://joss.theoj.org/papers/10.21105/joss.06504"
volume: 9
title: "NCDatasets.jl: a Julia package for manipulating netCDF data
sets"
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2017-2024: Alexander Barth.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
22 changes: 0 additions & 22 deletions LICENSE.md

This file was deleted.

6 changes: 3 additions & 3 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ uuid = "85f8d34a-cbdd-5861-8df4-14fed0d494ab"
keywords = ["netcdf", "climate and forecast conventions", "oceanography", "meteorology", "climatology", "opendap"]
license = "MIT"
desc = "Load and create NetCDF files in Julia"
version = "0.14.3"
version = "0.14.6"

[deps]
CFTime = "179af706-886a-5703-950a-314cd64e0468"
Expand All @@ -19,8 +19,8 @@ Printf = "de0858da-6303-5e67-8744-51eddeeeb8d7"
CFTime = "0.1.1"
CommonDataModel = "0.3.4"
DataStructures = "0.17, 0.18"
DiskArrays = "0.4"
NetCDF_jll = "=400.701.400, =400.702.400, =400.902.5, =400.902.208, =400.902.209"
DiskArrays = "0.4.5"
NetCDF_jll = "=400.701.400, =400.702.400, =400.902.5, =400.902.208, =400.902.209, =400.902.211"
NetworkOptions = "1.2"
julia = "1.6"

Expand Down
128 changes: 68 additions & 60 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# NCDatasets

[![Build Status](https://github.com/Alexander-Barth/NCDatasets.jl/workflows/CI/badge.svg)](https://github.com/Alexander-Barth/NCDatasets.jl/actions)
[![codecov.io](http://codecov.io/github/Alexander-Barth/NCDatasets.jl/coverage.svg?branch=master)](http://app.codecov.io/github/Alexander-Barth/NCDatasets.jl?branch=master)
[![codecov](https://codecov.io/github/Alexander-Barth/NCDatasets.jl/graph/badge.svg?token=SXpIBsfRrI)](https://codecov.io/github/Alexander-Barth/NCDatasets.jl)
[![documentation stable](https://img.shields.io/badge/docs-stable-blue.svg)](https://alexander-barth.github.io/NCDatasets.jl/stable/)
[![documentation dev](https://img.shields.io/badge/docs-dev-blue.svg)](https://alexander-barth.github.io/NCDatasets.jl/dev/)
[![DOI](https://joss.theoj.org/papers/10.21105/joss.06504/status.svg)](https://doi.org/10.21105/joss.06504)


`NCDatasets` allows one to read and create netCDF files.
Expand Down Expand Up @@ -35,11 +36,59 @@ Pkg.add("NCDatasets")

This Manual is a quick introduction in using NCDatasets.jl. For more details you can read the [stable](https://alexander-barth.github.io/NCDatasets.jl/stable/) or [latest](https://alexander-barth.github.io/NCDatasets.jl/latest/) documentation.

* [Create a netCDF file](#create-a-netcdf-file)
* [Explore the content of a netCDF file](#explore-the-content-of-a-netcdf-file)
* [Load a netCDF file](#load-a-netcdf-file)
* [Create a netCDF file](#create-a-netcdf-file)
* [Edit an existing netCDF file](#edit-an-existing-netcdf-file)

## Create a netCDF file

The following gives an example of how to create a netCDF file by defining dimensions, variables and attributes.

```julia
using NCDatasets
using DataStructures: OrderedDict
# This creates a new NetCDF file called file.nc.
# The mode "c" stands for creating a new file (clobber)
ds = NCDataset("file.nc","c")

# Define the dimension "lon" and "lat" with the size 100 and 110 resp.
defDim(ds,"lon",100)
defDim(ds,"lat",110)

# Define a global attribute
ds.attrib["title"] = "this is a test file"

# Define the variables temperature with the attribute units
v = defVar(ds,"temperature",Float32,("lon","lat"), attrib = OrderedDict(
"units" => "degree Celsius",
"scale_factor" => 10,
))

# add additional attributes
v.attrib["comments"] = "this is a string attribute with Unicode Ω ∈ ∑ ∫ f(x) dx"

# Generate some example data
data = [Float32(i+j) for i = 1:100, j = 1:110];

# write a single column
v[:,1] = data[:,1];

# write a the complete data set
v[:,:] = data;

close(ds)
```

It is also possible to create the dimensions, the define the variable and set its value with a single call to `defVar`:

```julia
using NCDatasets
ds = NCDataset("/tmp/test2.nc","c")
data = [Float32(i+j) for i = 1:100, j = 1:110]
v = defVar(ds,"temperature",data,("lon","lat"))
close(ds)
```
## Explore the content of a netCDF file

Before reading the data from a netCDF file, it is often useful to explore the list of variables and attributes defined in it.
Expand All @@ -63,24 +112,28 @@ while to get the global attributes you can do:
```julia
ds.attrib
```
which produces a listing like:

`NCDataset("file.nc")` produces a listing like:

```
Dataset: file.nc
Group: /
Dimensions
time = 115
lon = 100
lat = 110
Variables
time (115)
Datatype: Float64
Dimensions: time
temperature (100 × 110)
Datatype: Float32 (Float32)
Dimensions: lon × lat
Attributes:
calendar = gregorian
standard_name = time
units = days since 1950-01-01 00:00:00
[...]
units = degree Celsius
scale_factor = 10
comments = this is a string attribute with Unicode Ω ∈ ∑ ∫ f(x) dx
Global attributes
title = this is a test file
```

## Load a netCDF file
Expand All @@ -89,7 +142,7 @@ Loading a variable with known structure can be achieved by accessing the variabl

```julia
# The mode "r" stands for read-only. The mode "r" is the default mode and the parameter can be omitted.
ds = NCDataset("/tmp/test.nc","r")
ds = NCDataset("file.nc","r")
v = ds["temperature"]

# load a subset
Expand All @@ -99,7 +152,7 @@ subdata = v[10:30,30:5:end]
data = v[:,:]

# load all data ignoring attributes like scale_factor, add_offset, _FillValue and time units
data2 = v.var[:,:]
data2 = v.var[:,:];


# load an attribute
Expand All @@ -110,7 +163,7 @@ close(ds)
In the example above, the subset can also be loaded with:

```julia
subdata = NCDataset("/tmp/test.nc")["temperature"][10:30,30:5:end]
subdata = NCDataset("file.nc")["temperature"][10:30,30:5:end]
```

This might be useful in an interactive session. However, the file `test.nc` is not directly closed (closing the file will be triggered by Julia's garbage collector), which can be a problem if you open many files. On Linux the number of opened files is often limited to 1024 (soft limit). If you write to a file, you should also always close the file to make sure that the data is properly written to the disk.
Expand All @@ -123,52 +176,7 @@ data = NCDataset(filename,"r") do ds
end # ds is closed
```

## Create a netCDF file

The following gives an example of how to create a netCDF file by defining dimensions, variables and attributes.

```julia
using NCDatasets
using DataStructures
# This creates a new NetCDF file /tmp/test.nc.
# The mode "c" stands for creating a new file (clobber)
ds = NCDataset("/tmp/test.nc","c")

# Define the dimension "lon" and "lat" with the size 100 and 110 resp.
defDim(ds,"lon",100)
defDim(ds,"lat",110)

# Define a global attribute
ds.attrib["title"] = "this is a test file"

# Define the variables temperature with the attribute units
v = defVar(ds,"temperature",Float32,("lon","lat"), attrib = OrderedDict(
"units" => "degree Celsius"))

# add additional attributes
v.attrib["comments"] = "this is a string attribute with Unicode Ω ∈ ∑ ∫ f(x) dx"

# Generate some example data
data = [Float32(i+j) for i = 1:100, j = 1:110]

# write a single column
v[:,1] = data[:,1]

# write a the complete data set
v[:,:] = data

close(ds)
```

It is also possible to create the dimensions, the define the variable and set its value with a single call to `defVar`:

```julia
using NCDatasets
ds = NCDataset("/tmp/test2.nc","c")
data = [Float32(i+j) for i = 1:100, j = 1:110]
v = defVar(ds,"temperature",data,("lon","lat"))
close(ds)
```


## Edit an existing netCDF file
Expand All @@ -178,7 +186,7 @@ to open it with the `"a"` option. Here, for example, we add a global attribute *
file created in the previous step.

```julia
ds = NCDataset("/tmp/test.nc","a")
ds = NCDataset("file.nc","a")
ds.attrib["creator"] = "your name"
close(ds);
```
Expand Down
Loading

0 comments on commit 12b35ad

Please sign in to comment.