A study on the method of eliminating duplication of ocean temperature and salinity data

Abstract:
Duplicate data introduced during data collection, transmission, exchange, and management will lead to uncertainty in the total amount of data and cause unreliability of the whole data set, resulting in erroneous results of statistical analysis of marine characteristics. In this paper, based on extensive data processing practices, the author first clarifies the source and type of temperature and salinity duplicate data, then proposes a series of threshold for identifying repeated data from different instruments, and establishes a process for identifying and eliminating duplicate data. In this way, duplicated data in the WOD data are effectively removed. This operational process can effectively remove the duplicates of temperature and salinity data and improve the application value of the dataset, which can also be applied to other types of data.