/*! \page page_calibration Device Calibration and Frontend Correction

\tableofcontents

\section calibration_self Self-Calibration

UHD software comes with several self-calibration utilities for
minimizing IQ imbalance and DC offset. These utilities perform
calibration sweeps using transmit leakage into the receive path (special
equipment is not required). The results from a calibration are written
to a file in the user's home directory. UHD software will
automatically apply corrections at runtime when the user re-tunes the
daughterboard LO. Calibration results are specific to an individual RF
board.

<b>Note:</b> When a calibration table is present, and the user wishes to
override the calibration settings through the API: the user should
re-apply the desired setting every time the LO is re-tuned.

UHD software comes with the following calibration utilities:

-   **uhd_cal_rx_iq_balance:** - minimizes RX IQ imbalance vs. LO
    frequency
-   **uhd_cal_tx_dc_offset:** - minimizes TX DC offset vs. LO
    frequency
-   **uhd_cal_tx_iq_balance:** - minimizes TX IQ imbalance vs. LO
    frequency

The following RF frontends are supported by the self-calibration
utilities:

-   RFX Series transceiver boards
-   WBX Series transceiver boards
-   SBX Series transceiver boards
-   CBX Series transceiver boards
-   UBX Series transceiver boards
-   OBX Series transceiver boards
-   USRP N320

The **USRP X420** with its large bandwidth of 1 GHz is not supported by
the aforementioned calibration utilities. For this device, the
`uhd_iq_dc_correction.py` utility needs to be used for both IQ imbalance
and DC offset correction.

\subsection calibration_self_utils Calibration Utilities

UHD software installs the calibration utilities into
`<install-path>/bin`. **Disconnect** any external hardware from the
RF antenna ports, and run the following from the command line. Each
utility will take several minutes to complete:

    uhd_cal_rx_iq_balance --verbose --args=<optional device args>
    uhd_cal_tx_iq_balance --verbose --args=<optional device args>
    uhd_cal_tx_dc_offset --verbose --args=<optional device args>

See the output given by `--help` for more advanced options, such as
manually choosing the frequency range and step size for the sweeps.

<b>Note:</b> Your daughterboard needs a serial number to run a calibration
utility. Some older daughterboards may not have a serial number. If this
is the case, run the following command to burn a serial number into the
daughterboard's EEPROM:

    <install dir>/lib/uhd/utils/usrp_burn_db_eeprom --ser=<desired serial> --args=<optional device args>

\subsection calibration_iq_dc_utils IQ Imbalance and DC Offset Correction Utilities

Currently, for X420 only, UHD installs the **uhd_iq_dc_correction.py**
utility into `<Python path>/site-packages/uhd/utils/`. In addition to
the requirements of UHD, this script requires the Python packages
`scipy` and `tqdm`. As with the other calibration utilities, it is
recommended to disconnect any external signals from the RF antenna
ports before running the utility and to let the device warm up for
a few minutes. By default, the utility will perform an RX IQ imbalance and DC Offset
correction only. By adding the `--tx` argument, TX IQ imbalance and DC offset corrections can be enabled
as well. The utility will take several minutes to complete. See the output
given by `--help` for more advanced options, such as manually choosing
the frequency range and step size for the sweeps or switching to a
different than the default master clock rate.

    python3 uhd_iq_dc_correction.py --args=<optional device args> -c <chan> [--tx --sample_rate=<rate>]

It is recommended to perform the corrections using a 10 or 100 GbE connection between USRP and host computer to minimize
the measurement time. If that is not possible, the corrections can still be executed using
the 1 GbE connection, too, but the additional --rj45 parameter must be
added to disable optimizations that are specific to higher-speed connections.

\subsection calibration_data Calibration Data

By default, calibration files are stored in the user's home/application
directory (`$XDG_DATA_HOME`):

-   **Linux:** `${HOME}/.local/share/uhd/cal/`
-   **Windows:** `%LOCALAPPDATA%\uhd\cal\`

Calibration files are binary files with a `.cal` file extension.

If you would like to specify a custom directory, you can do so with the
`$UHD_CAL_DATA_PATH` environment variable.

Calibration files can easily be moved from one machine to another by copying the
"cal" directory, or individual files therein.  Re-running a calibration utility
will replace the existing calibration file. The old calibration file will be
renamed so it may be recovered by the user.

\subsection modify_cal_data Modify Calibration Data

There might be reasons to analyse or modify the calibration data outside UHD's
calibration process. Because the data is stored using FlatBuffers this can be
done without relying on UHD. UHD provides all FlatBuffers schema files in
`<install-path>/share/uhd/cal`.

First, install FlatBuffers. The package can be obtained from
https://google.github.io/flatbuffers/.
Once installed, `.cal` files can be converted to JSON using

	flatc --strict-json -t <install dir>/share/uhd/cal/<foo>_cal.fbs -- <data>.cal

where `<foo>_cal.fbs` is the scheme file used for the data,
e.g. `pwr_cal.fbs` for power calibration. `data.cal` is a calibration file in
your working directory. This will generate a `<data>.json` in the same
directory.

The JSON data can be converted back to binary using

	flatc -b <install dir>/include/uhd/cal/<foo>_cal.fbs <data>.json

This generates a `<data>.cal` that can be read by the calibration routines
of UHD.

\subsection calibration_data_csv Converting UHD 3.x calibration data to UHD 4

Older versions of UHD used a CSV-based format for storing calbration data for
IQ imbalance and DC offset correction on some devices (e.g., X300, N200
motherboards and WBX/SBX/CBX/UBX/OBX daughterboards).

Going forward, all calibration data is stored as binary, to facilitate storing
it on device's flash memory, among other reasons. Running the `uhd_cal_*`
utilities will automatically generate the calibration data in the new format.

To convert existing calbration data to the new format, use the convert_cal_data.py
utility. By default, it will convert all existing data. Use `convert_cal_data.py --help`
to get a full list of command line options.
The tool is installed with the other utilities, for example into `/usr/share/lib/uhd/utils`,
depending on your OS and CMake settings.


\subsection ignore_cal_file Ignoring Calibration Files

At runtime, the user can choose to ignore a daughterboard's calibration file by
adding "ignore-cal-file" to the arguments. With the UHD API, it can be done as
follows:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.cpp}
    auto usrp = uhd::usrp::multi_usrp::make("type=x300,ignore-cal-file=1");
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Using tx_waveforms as an example, the user can apply this argument as follows:

    tx_waveforms --args="addr=192.168.10.2,ignore-cal-file=1" --freq=100e6 --rate=1e6


\section calibration_fe_corr Frontend Corrections

The calibrations for IQ imbalance and DC offset compensation rely on frontend
correction logic that is located in the FPGA.

Note that USRP E310, E320, N300, N310 and B200-Series use a dedicated RFIC which does
its own calibration. For those, any calibrations are very device-specific and
are not covered in this section.

\subsection calibration_fe_corr_dc_offset Single-Tap DC Offset Compensation

A DC offset is a fixed voltage that is permanently present on the signal of
interest. It is an additive error, meaning if the signal of interest is x(t),
the actual, observed signal y(t) includes a constant offset:

\f[
    y(t) = x(t) + D
\f]

If the constant value D is known, it can simply be subtracted again. For this
reason, the DC offset compensation consists of a simple adder. The calibration
tool `uhd_cal_tx_dc_offset` will estimate the value D' which minimizes the DC
offset for various frequencies (i.e., D' is approximately equal to -D).

To program the adder, set the DC offset correction value using these API calls
(all USRPs, except USRP X420):

- uhd::usrp::multi_usrp::set_tx_dc_offset()
- uhd::usrp::multi_usrp::set_rx_dc_offset()
- uhd::rfnoc::radio_control::set_tx_dc_offset()
- uhd::rfnoc::radio_control::set_rx_dc_offset()

For RX DC offset compensation, the estimation of D can be difficult. Therefore,
the RX DC offset compensation also includes an automatic mode, which acts as a
notch filter around DC. It is implemented as a single-tap IIR filter with the
following difference equation:

\f[
    y[k] = x[k] - \alpha * y[k-1]
\f]

The values for \f$\alpha\f$ are device-dependent, but the default value is \f$2^{-20}\f$.

To enable the automatic DC offset correction, call one of these API calls with
an argument of 'true' (all USRPs, except USRP X420):

- uhd::usrp::multi_usrp::set_rx_dc_offset()
- uhd::rfnoc::radio_control::set_rx_dc_offset()

DC offset is often caused by LO leakage. It is therefore advised to offset-tune
the radio to avoid having the LO in the band of interest. The DC offset
compensation will not only remove LO spurs, but also affect the signal of
interest if the LO is within the band of interest.

Superheterodyne receivers such as the TwinRX do usually not require DC offset
correction values.

Because IQ impairments and DC offset measurements are similar in the way how
they are measured, **in X420**, the same utility is used for both. The API
calls described above do not apply to X420. Instead, after running the
`uhd_iq_dc_correction.py` utility, the correction values are automatically applied
at runtime. Passing them manually is not possible.

\subsection calibration_fe_corr_iq_offset Single-Point IQ offset correction

In order to correct IQ imbalance, the user can specify two real values A and B.
In the FPGA, the complex signal is modified according to the following equation:

\f[
\begin{pmatrix}
I' \\
Q'
\end{pmatrix}
=
\begin{pmatrix}
A/64+1 & 0 \\
B/64   & 1 \\
\end{pmatrix}
\begin{pmatrix}
I \\
Q
\end{pmatrix}
\f]

This is a normalized version of the usual correction matrix for IQ imbalance
which is easier to implement in the FPGA. Because the top-left value of this
matrix depends on the magnitude error of the IQ imbalance, and the bottom row
depends on the phase error, the value "A" is often referred to as the "magnitude
correction value", and the value "B" is referred to as the "phase correction value".
These values are however not identical to the actual phase and magnitude error
caused by IQ imbalance.

The `uhd_cal_tx_iq_balance` and `uhd_cal_rx_iq_balance` tools will simply search
for A and B values which minimize the error caused by IQ imbalance.

Notes:
- The API calls uhd::usrp::multi_usrp::set_rx_iq_balance() and
  uhd::usrp::multi_usrp::set_tx_iq_balance() take complex numbers as a correction
  value, which is generated by `std::complex<double>(A, B)`. The complex math is
  not necessary, given the correction algorithm, but the argument was chosen due
  to its symmetry to the DC offset correction APIs, and also because treating the
  correction value as a complex number lets it be handled and stored easily and
  exactly as with the DC offset correction.
- This correction algorithm is suboptimal for high bandwidths, albeit better
  than nothing.

IQ imbalance is caused by the inphase and quadrature paths of a direct-conversion
receiver being slightly different in amplitude as well as their phase not being
exactly 90 degrees apart (due to analog components). It is therefore not
required for superheterodyne architectures like the TwinRX.

\subsection calibration_fe_dc_iq DC Offset and IQ Imbalance Correction (X420)

The calibration files generated from the uhd_iq_dc_correction utility are
compensating the DC offset as described in \ref calibration_fe_corr_dc_offset.

The IQ imbalance correction is based on the single-point method described before with an additional compensation of the
time delay of the Q channel.

The time delay is compensated using a fractional-delay FIR filter.
The coefficients \f$h\f$ are calculated from the following equation, where \f$d\f$ is the
desired group delay of the filter (centered by the number of coefficients) and
\f$F_s\f$ is the sample rate.

\f[
h(i) = sinc((i-d) - t_{offset} * F_s)
\f]

The single point IQ correction is using the formula below, where the linear
gain is \f$g\f$ and the phase skew is \f$\phi\f$.

\f[
\begin{pmatrix}
I' \\
Q'
\end{pmatrix}
=
\begin{pmatrix}
1 & 0 \\
-tan(\phi) & \frac{1}{g \cdot cos(\phi)}\\
\end{pmatrix}
\begin{pmatrix}
I \\
Q
\end{pmatrix}
\f]

For the Q inline path
(\f$Q \rightarrow Q'\f$) the respective factor from the correction matrix and the coefficients \f$h\f$  from the
FIR filter above are combined into a single FIR filter.

*/
// vim:ft=doxygen:
