HDF5 metadata layout

Overview

Diffraction data are written on the server by Jungfraujoch (jfjoch_writer), producing NeXus/NXmx-style HDF5 files. After acquisition, the Jungfrau GUI (JFGui) optionally post-processes the written HDF5 and adds/updates electron-diffraction/TEM-specific metadata (beam fit, TEM state, stage motion, apertures, etc.).

This page documents:

1. The baseline file structure produced by Jungfraujoch (master + data files).

2. The extra/overwritten fields written by the JFGui (this project).

3. Conventions (units, datatypes, and where values come from).

References

• JFJOCH_WRITER — Jungfraujoch writer file structure (upstream).

• NXmx — NeXus NXmx application definition (schema reference).

• NXmx_xml — NXmx NXDL definition file (formal schema).

Files produced by Jungfraujoch

Jungfraujoch writes a master file (high-level metadata + links) and one or more data files (the actual image stacks). From the GUI perspective, the important point is:

The diffraction frames are stored in the data files; the master file aggregates metadata and provides convenient links to the data.

Master file vs data file layout

Master file (overview)

The master file (*_master.h5) contains the NeXus entry and metadata groups (e.g. /entry/instrument, /entry/sample, /entry/source). It also contains an /entry/data group that acts as a link table to the individual data files:

• /entry/data/data_000001, data_000002, … are links to frame datasets stored in the corresponding *_data_*.h5 files.

• The master file therefore references the frames but does not store them as a single contiguous dataset.

Data file (frames)

Each data file (e.g. *_data_000034.h5) stores the actual frames as a single dataset:

• /entry/data/data → shape (nframes, ny, nx)

A non-exhaustive view of the relevant groups is shown below (see JFJOCH_WRITER for the full structure):

File	Path	Description
Master	/entry/data/data_000001 … data_000XYZ	Links to per-run data files / frame datasets
Data	/entry/data/data	Raw image data (stack: nframes × ny × nx)
Master (optional)	/entry/MX/*	Spot finding / indexing auxiliaries (optional)
Master (optional)	/entry/azint/*	Azimuthal integration results (optional)
Master (optional)	/entry/user/*	User-provided metadata (optional; depends on acquisition configuration)

JFGui metadata injection

JFGui post-processes the master HDF5 file after acquisition and only updates metadata (i.e. it does not modify the diffraction frames under /entry/data). See Overwrite semantics for details on the create-or-replace behavior.

The injected metadata are grouped below by subsystem.

1) Detector group

Paths under /entry/instrument/detector:

Path	Type	Unit	Description
/entry/instrument/detector/detector_name	string		Human-readable detector identifier (site-specific)
/entry/instrument/detector/beam_center_x	int	pixel	Beam center X from beam fitting (JFGui)
/entry/instrument/detector/beam_center_y	int	pixel	Beam center Y from beam fitting (JFGui)
/entry/instrument/detector/detector_distance	uint64	mm	Calibrated sample-to-detector distance from LUT (see note).
/entry/instrument/detector/framerate	uint64	Hz	Detector frame rate (fixed for JUNGFRAU here)
/entry/instrument/detector/count_threshold_in_keV	uint64	keV	JUNGFRAU threshold used during acquisition

Paths under /entry/instrument/detector/detectorSpecific:

Path	Type	Unit	Description
/entry/instrument/detector/detectorSpecific/element	string		Sensor material label (e.g. Si)
/entry/instrument/detector/detectorSpecific/software_version_gui	string		JFGui version tag (JF_GUI/<tag>)
/entry/instrument/detector/detectorSpecific/gui_commit_hash	string		Git commit hash of the GUI build

2) Source group

Path	Type	Unit	Description
/entry/source/probe	string		Set to electron (ED)

Note

NeXus/NXmx is historically rooted in X-ray crystallography, but electron-diffraction pipelines commonly set probe=electron when writing NXmx-like files.

3) Optics (TEM state + beam characterization)

These are written under /entry/instrument/optics:

Path	Type	Unit	Description
.../info_acquisition_date_time	string		Timestamp of GUI metadata update
.../microscope_name	string		TEM model string
.../accelerationVoltage	float	kV	HT value (fallback to 200 kV on read error)
.../accelerationVoltage_readout	uint16		HT readout status (1=valid, 0=invalid)
.../wavelength	float	Å	Computed from acceleration voltage (relativistic electron wavelength)
.../magnification	uint16		TEM Magnification value readback (Imaging mode)
.../distance_nominal	uint16		TEM Magnification value readback (Diffraction mode)
.../end_tilt_angle	float	deg	Final TX tilt angle (stage readback)
.../spot_size	uint16		Spot size index (+1 stored). Range is 1-8
.../alpha_angle	uint16		Alpha index (+1 stored). Range is 1-9

Apertures:

Path	Type	Unit	Description
.../CL_ID	uint16	(index)	Condenser aperture hole number (0=open, 1–4=hole index).
.../CL_size	string	µm	Condenser aperture diameter from LUT mapping (CL_ID → size in µm).
.../CL_position_x	uint16	(device counts)	Condenser aperture centering X
.../CL_position_y	uint16	(device counts)	Condenser aperture centering Y
.../SA_ID	uint16	(index)	Selected-area aperture hole number (0=open, 1–4=hole index).
.../SA_size	string	µm	Selected-area (SA) aperture size label
.../SA_position_x	uint16	(device counts)	Selected-area (SA) aperture centering X
.../SA_position_y	uint16	(device counts)	Selected-area (SA) aperture centering Y

Note

Aperture positions are reported in microscope controller coordinates (“device counts”). In JEOL TEMExt-style interfaces these are typically 12-bit values (0–4095).

Lens/deflector state (raw readbacks):

Path	Type	Unit	Description
.../brightness	uint32	(device counts)	Raw value of CL3 lens setting (0–65535)
.../diff_focus	uint32	(device counts)	Raw value of IL1 lens setting (0–65535)
.../il_stigm_x	uint32	(device counts)	Raw value of X component of Intermediate Stigmator (IL Stig) deflector (0–65535)
.../il_stigm_y	uint32	(device counts)	Raw value of Y component of IL Stig (0–65535)
.../pl_align_x	uint32	(device counts)	Raw value of X component of the Projector Shift (PLA) deflector (0–65535)
.../pl_align_y	uint32	(device counts)	Raw value of Y component of the PLA (0–65535)

Beam fit / illumination (GUI-derived):

Path	Type	Unit	Description
.../optical_axis_center_x	float	pixel	Optical axis center X on detector (from LUT/config).
.../optical_axis_center_y	float	pixel	Optical axis center Y on detector (from LUT/config).
.../beam_width_sigmax	float	pixel	Beam width sigma X from beam fit
.../beam_width_sigmay	float	pixel	Beam width sigma Y from beam fit
.../beam_ellipse_angle	float	deg	Beam ellipse angle from fit
.../beam_illumination_pa_per_cm2_detector	float	pA/cm²	Estimated illumination at detector plane
.../beam_illumination_e_per_A2_sample	float	e⁻/Ų	Dose estimate at sample plane

4) Stage (position + rotation during collection)

Written under /entry/instrument/stage.

Path	Type	Unit	Description
.../stage_x	float	µm	Stage X (converted from TEM readback)
.../stage_y	float	µm	Stage Y (converted from TEM readback)
.../stage_z	float	µm	Stage Z (converted from TEM readback)
.../stage_xyz_unit	string		µm
.../stage_tx_speed_ID	float	(index)	Speed selector index from TEM
.../velocity_data_collection	float	deg/s	Nominal rotation speed from lookup table
.../stage_tx_axis	float[3]	(vector)	Rotation axis unit vector (XDS convention) from LUT (ht_voltage and magnification regime).

If rotation angle samples were recorded (rotations_angles not None), additional datasets are stored:

Path	Type	Unit	Description
.../stage_tx_start	float	deg	Start angle from recorded samples
.../stage_tx_end	float	deg	End angle from recorded samples
.../stage_tx_speed_measured	float	deg/s	Mean measured angular velocity from samples
.../stage_tx_speed_measured_std	float	deg/s	Std dev of measured angular velocity
.../stage_tx_speed_unit	string		deg/s
.../stage_tx_record	array	(time,deg)	Raw record (as stored by GUI)

5) CIF convenience block (downstream crystallography tooling)

Written under /entry/cif as simple text fields to help downstream export.

Path	Type	Unit	Description
/entry/cif/_diffrn_ambient_temperature	string	K	Fixed-format CIF temperature formatted as text
/entry/cif/_diffrn_radiation_wavelength	string	Å	Wavelength formatted as text
/entry/cif/_diffrn_radiation_probe	string		electron
/entry/cif/_diffrn_radiation_type	string		CIF label (monochromatic beam)
/entry/cif/_diffrn_source	string		TEM source
/entry/cif/_diffrn_source_type	string		TEM model (JEOL JEM2100Plus)
/entry/cif/_diffrn_source_voltage	string	kV	HT formatted as text
/entry/cif/_diffrn_measurement_device_type	string		Holder / goniometer (e.g. single axis tomography holder)
/entry/cif/_diffrn_detector	string		Detector class (e.g. hybrid pixel area detector)
/entry/cif/_diffrn_detector_type	string		Detector model (e.g. JUNGFRAU)
/entry/cif/_diffrn_detector_area_resol_mean	string	1/mm	Convenience value, computed as 1/pixel_size_mm

Notes

LUT-derived calibration values

Some metadata are not direct TEM readbacks; they are calibrated using lookup tables stored in jfgui2_config.json (e.g. detector distance, aperture sizes, rotation axis, optical-axis center).

• detector_distance is interpolated from LUT points as a function of DIFF nominal and HT.

• CL_size / SA_size map aperture IDs to diameters in µm.

• stage_tx_axis provides an XDS-style rotation-axis unit vector keyed by HT (and magnification regime).

• optical_axis_center_{x,y} are detector pixel coordinates defined for the current setup.

Overwrite semantics

JFGui uses “delete then recreate” for each dataset. This is simple, deterministic, and ensures the GUI-written metadata takes precedence. However, it also discards:

• existing dataset attributes,

• chunking/compression,

• original dtype choices.