icedb::Tables::CanHaveTables Class Reference

The virtual base class used in all objects that can contain tables (groups and datasets). More...

#include <Table.hpp>

Inheritance diagram for icedb::Tables::CanHaveTables:

Public Member Functions
	~CanHaveTables ()
std::set< std::string >	getTableNames () const
	Lists all table names that are children of this object. More...
bool	doesTableExist (const std::string &tableName) const
	Does a table exist with the given name. More...
void	unlinkTable (const std::string &tableName)
	Unlink a table. In HDF5, this is not the same as erasing a table, which never actually happens. More...
Table::Table_Type	openTable (const std::string &tableName)
std::vector< size_t >	getChunkStrategy (const std::vector< size_t > &dims)
	The default chunking strategy for this table. Used for storage i/o speed, and for compression. More...
template<class DataType >
Table::Table_Type	createTable (const std::string &tableName, const std::vector< size_t > &dims, const std::vector< size_t > *chunks=nullptr)
	Create a table. More...
template<class DataType >
Table::Table_Type	createTable (const std::string &tableName, std::initializer_list< size_t > dims, const std::vector< size_t > *chunks=nullptr)
	Create a table. More...
template<class DataType >
Table::Table_Type	createTable (const std::string &tableName, std::initializer_list< size_t > dims, std::initializer_list< DataType > data, const std::vector< size_t > *chunks=nullptr)
	Create a table and writes initial data. Used with small tables. More...
template<class DataType >
Table::Table_Type	createTable (const std::string &tableName, std::initializer_list< size_t > dims, const std::vector< DataType > &data, const std::vector< size_t > *chunks=nullptr)
	Create a table and sets the table's initial data. More...
template<class DataType >
Table::Table_Type	createTable (const std::string &tableName, std::initializer_list< size_t > dims, const gsl::span< DataType > &data, const std::vector< size_t > *chunks=nullptr)
	Create a table and sets the table's initial data. More...
template<class DataType >
Table::Table_Type	createTable (const std::string &tableName, std::initializer_list< size_t > dims, const gsl::span< const DataType > &data, const std::vector< size_t > *chunks=nullptr)
	Create a table and sets the table's initial data. More...

Protected Member Functions
	CanHaveTables ()
	Trivial constructor used when CanHaveTables is a virtual base class. More...
virtual void	_setTablesParent (std::shared_ptr< H5::Group > obj)=0
	CanHaveTables needs post-constructor setup. This sets the base HDF5 object that gets manipulated. More...
virtual std::shared_ptr< H5::Group >	_getTablesParent () const =0
	Gets the base HDF5 object that is manipulated. More...

Private Member Functions
bool	valid () const
	Checks that the class is well-formed, constructed and activated. CanHaveTables needs post-constructor setup. More...
void	_createTable (const std::string &tableName, const std::type_index &type, const std::vector< size_t > &dims, const std::vector< size_t > &chunks)
	Internal function to create a table. Does not write data. More...

Detailed Description

The virtual base class used in all objects that can contain tables (groups and datasets).

Definition at line 126 of file Table.hpp.

Constructor & Destructor Documentation

CanHaveTables()

icedb::Tables::CanHaveTables::CanHaveTables ( )

protected

Trivial constructor used when CanHaveTables is a virtual base class.

Definition at line 245 of file Tables.cpp.

{}

~CanHaveTables()

icedb::Tables::CanHaveTables::~CanHaveTables

(

)

Definition at line 246 of file Tables.cpp.

{}

Member Function Documentation

_createTable()

void icedb::Tables::CanHaveTables::_createTable ( const std::string &  tableName, const std::type_index &  type_id, const std::vector< size_t > &  dims, const std::vector< size_t > &  chunks  )

private

Internal function to create a table. Does not write data.

Parameters

tableName	is the name of the table.
type	is the type of the table.
dims	are the dimensions of the table.
chunks	are the HDF5 chunking parameters, used when writing a table.

Note

Must ensure NetCDF-4 compatability.

Definition at line 250 of file Tables.cpp.

References icedb::fs::hdf5::useZLIB(), and icedb::Tables::Table::valid().

        {
            Expects(valid());
            std::vector<hsize_t> hdata;
            for (const auto &d : dims) hdata.push_back(static_cast<hsize_t>(d));
            std::vector<hsize_t> cdata;
            for (const auto &c : chunks) cdata.push_back(static_cast<hsize_t>(c));

            //fs::hdf5::addDatasetArray
            /* Turn on creation order tracking. */
            //assert(H5Pset_attr_creation_order(plistid, H5P_CRT_ORDER_TRACKED | H5P_CRT_ORDER_INDEXED) >= 0);
            std::shared_ptr<H5::DSetCreatPropList> plist = std::make_shared<H5::DSetCreatPropList>();
            int fillvalue = -1;
            plist->setFillValue(H5::PredType::NATIVE_INT, &fillvalue);
            plist->setChunk(static_cast<int>(cdata.size()), cdata.data());
            if (fs::hdf5::useZLIB()) {
                plist->setDeflate(fs::hdf5::useZLIB());
            }
#if ICEDB_H5_CREATEPROPLIST_SETATTRCRTORDER == 1
            plist->setAttrCrtOrder(H5P_CRT_ORDER_TRACKED | H5P_CRT_ORDER_INDEXED);
#else
            {
                hid_t id = plist->getId();
                herr_t e = H5Pset_attr_creation_order(id, H5P_CRT_ORDER_TRACKED | H5P_CRT_ORDER_INDEXED);
                if (e < 0) throw;
            }
#endif

            if (type_id == typeid(uint64_t))fs::hdf5::createDatasetRaw<uint64_t, H5::Group>(_getTablesParent().get(), tableName.c_str(), dims, plist);
            else if (type_id == typeid(int64_t))fs::hdf5::createDatasetRaw<int64_t, H5::Group>(_getTablesParent().get(), tableName.c_str(), dims, plist);
            else if (type_id == typeid(uint32_t))fs::hdf5::createDatasetRaw<uint32_t, H5::Group>(_getTablesParent().get(), tableName.c_str(), dims, plist);
            else if (type_id == typeid(int32_t))fs::hdf5::createDatasetRaw<int32_t, H5::Group>(_getTablesParent().get(), tableName.c_str(), dims, plist);
            else if (type_id == typeid(uint16_t))fs::hdf5::createDatasetRaw<uint16_t, H5::Group>(_getTablesParent().get(), tableName.c_str(), dims, plist);
            else if (type_id == typeid(int16_t))fs::hdf5::createDatasetRaw<int16_t, H5::Group>(_getTablesParent().get(), tableName.c_str(), dims, plist);
            else if (type_id == typeid(uint8_t))fs::hdf5::createDatasetRaw<uint8_t, H5::Group>(_getTablesParent().get(), tableName.c_str(), dims, plist);
            else if (type_id == typeid(int8_t))fs::hdf5::createDatasetRaw<int8_t, H5::Group>(_getTablesParent().get(), tableName.c_str(), dims, plist);
            else if (type_id == typeid(float))fs::hdf5::createDatasetRaw<float, H5::Group>(_getTablesParent().get(), tableName.c_str(), dims, plist);
            else if (type_id == typeid(double))fs::hdf5::createDatasetRaw<double, H5::Group>(_getTablesParent().get(), tableName.c_str(), dims, plist);
            else if (type_id == typeid(char))fs::hdf5::createDatasetRaw<char, H5::Group>(_getTablesParent().get(), tableName.c_str(), dims, plist);
            else if (type_id == typeid(std::string))fs::hdf5::createDatasetRaw<std::string, H5::Group>(_getTablesParent().get(), tableName.c_str(), dims, plist);
            else throw(std::invalid_argument("Unhandled data type"));
        }

Here is the call graph for this function:

_getTablesParent()

virtual std::shared_ptr<H5::Group> icedb::Tables::CanHaveTables::_getTablesParent ( ) const

protectedpure virtual

Gets the base HDF5 object that is manipulated.

Implemented in icedb::Tables::CanHaveTables_impl.

_setTablesParent()

virtual void icedb::Tables::CanHaveTables::_setTablesParent ( std::shared_ptr< H5::Group >  obj )

protectedpure virtual

CanHaveTables needs post-constructor setup. This sets the base HDF5 object that gets manipulated.

Implemented in icedb::Tables::CanHaveTables_impl.

createTable() [1/6]

template<class DataType >

Table::Table_Type icedb::Tables::CanHaveTables::createTable ( const std::string &  tableName, const std::vector< size_t > &  dims, const std::vector< size_t > *  chunks = nullptr  )

inline

Create a table.

Parameters

DataType	is the type of the data
tableName	is the name of the table

Exceptions

if	a table with this name already exists
if	the base object is read only

Parameters

dims	are the dimensions of the table
chunks	is the size of each N-dimensional chunk, used for storage and compression. HDF5 writes each table as a group of chunked objects. When reading or writing, entire chunks are always read or written.

Returns

The new table.

Definition at line 172 of file Table.hpp.

                                                                                                                                              {
                Expects(!doesTableExist(tableName));
                std::vector<size_t> chunksGuess;
                if (!chunks) {
                    chunksGuess = getChunkStrategy(dims);
                    chunks = &chunksGuess;
                }
                _createTable(tableName, Data_Types::getType<DataType>(), dims, *chunks);
                return openTable(tableName);
            }

createTable() [2/6]

template<class DataType >

Table::Table_Type icedb::Tables::CanHaveTables::createTable ( const std::string &  tableName, std::initializer_list< size_t >  dims, const std::vector< size_t > *  chunks = nullptr  )

inline

Create a table.

Parameters

DataType	is the type of the data
tableName	is the name of the table

Exceptions

if	a table with this name already exists
if	the base object is read only

Parameters

dims	are the dimensions of the table
chunks	is the size of each N-dimensional chunk, used for storage and compression. HDF5 writes each table as a group of chunked objects. When reading or writing, entire chunks are always read or written.

Returns

The new table.

Definition at line 193 of file Table.hpp.

            {
                std::vector<size_t> vdims{ dims };
                auto tbl = createTable<DataType>(tableName, vdims, chunks);
                return tbl;
            }

createTable() [3/6]

template<class DataType >

Table::Table_Type icedb::Tables::CanHaveTables::createTable ( const std::string &  tableName, std::initializer_list< size_t >  dims, std::initializer_list< DataType >  data, const std::vector< size_t > *  chunks = nullptr  )

inline

Create a table and writes initial data. Used with small tables.

Parameters

DataType	is the type of the data
tableName	is the name of the table

Exceptions

if	a table with this name already exists
if	the base object is read only

Parameters

dims	are the dimensions of the table
data	are the initial data of the table
chunks	is the size of each N-dimensional chunk, used for storage and compression. HDF5 writes each table as a group of chunked objects. When reading or writing, entire chunks are always read or written.

Returns

The new table.

Definition at line 214 of file Table.hpp.

            {
                auto tbl = createTable<DataType>(tableName, dims, chunks);
                //std::vector<DataType> vdata{ data };
                tbl->template writeAll<DataType>(gsl::span<const DataType>(data.begin(), data.end()));
                return tbl;
            }

createTable() [4/6]

template<class DataType >

Table::Table_Type icedb::Tables::CanHaveTables::createTable ( const std::string &  tableName, std::initializer_list< size_t >  dims, const std::vector< DataType > &  data, const std::vector< size_t > *  chunks = nullptr  )

inline

Create a table and sets the table's initial data.

Parameters

DataType	is the type of the data
tableName	is the name of the table

Exceptions

if	a table with this name already exists
if	the base object is read only

Parameters

dims	are the dimensions of the table
data	are the data of the table
chunks	is the size of each N-dimensional chunk, used for storage and compression. HDF5 writes each table as a group of chunked objects. When reading or writing, entire chunks are always read or written.

Returns

The new table.

Definition at line 237 of file Table.hpp.

            {
                auto tbl = createTable<DataType>(tableName, dims, chunks);
                tbl->template writeAll<DataType>(data);
                return tbl;
            }

createTable() [5/6]

template<class DataType >

Table::Table_Type icedb::Tables::CanHaveTables::createTable ( const std::string &  tableName, std::initializer_list< size_t >  dims, const gsl::span< DataType > &  data, const std::vector< size_t > *  chunks = nullptr  )

inline

Create a table and sets the table's initial data.

Parameters

DataType	is the type of the data
tableName	is the name of the table

Exceptions

if	a table with this name already exists
if	the base object is read only

Parameters

dims	are the dimensions of the table
data	are the data of the table
chunks	is the size of each N-dimensional chunk, used for storage and compression. HDF5 writes each table as a group of chunked objects. When reading or writing, entire chunks are always read or written.

Returns

The new table.

Definition at line 259 of file Table.hpp.

            {
                auto tbl = createTable<DataType>(tableName, dims, chunks);
                tbl->template writeAll<DataType>(data);
                return tbl;
            }

createTable() [6/6]

template<class DataType >

Table::Table_Type icedb::Tables::CanHaveTables::createTable ( const std::string &  tableName, std::initializer_list< size_t >  dims, const gsl::span< const DataType > &  data, const std::vector< size_t > *  chunks = nullptr  )

inline

Create a table and sets the table's initial data.

Parameters

DataType	is the type of the data
tableName	is the name of the table

Exceptions

if	a table with this name already exists
if	the base object is read only

Parameters

dims	are the dimensions of the table
data	are the data of the table
chunks	is the size of each N-dimensional chunk, used for storage and compression. HDF5 writes each table as a group of chunked objects. When reading or writing, entire chunks are always read or written.

Returns

The new table.

Definition at line 281 of file Table.hpp.

            {
                auto tbl = createTable<DataType>(tableName, dims, chunks);
                tbl->template writeAll<DataType>(data);
                return tbl;
            }

doesTableExist()

bool icedb::Tables::CanHaveTables::doesTableExist

(

const std::string &

tableName

)

const

Does a table exist with the given name.

Definition at line 309 of file Tables.cpp.

                                                                           {
            const auto tnames = getTableNames();
            if (tnames.count(tableName)) return true;
            return false;
        }

getChunkStrategy()

std::vector<size_t> icedb::Tables::CanHaveTables::getChunkStrategy ( const std::vector< size_t > &  dims )

inline

The default chunking strategy for this table. Used for storage i/o speed, and for compression.

Todo:

Use a more intelligent strategy

Returns

the ideal size of each chunk of the dataset.

Definition at line 158 of file Table.hpp.

                                                                                     {
                return dims;
            }

getTableNames()

std::set< std::string > icedb::Tables::CanHaveTables::getTableNames

(

)

const

Lists all table names that are children of this object.

Definition at line 297 of file Tables.cpp.

References icedb::fs::hdf5::getGroupMembersTypes(), and icedb::Tables::Table::valid().

                                                             {
            Expects(valid());
            auto base = _getTablesParent();
            auto objs = fs::hdf5::getGroupMembersTypes(*(base.get()));
            std::set<std::string> res;
            for (const auto &o : objs)
            {
                if (o.second == H5G_obj_t::H5G_DATASET) res.insert(o.first);
            }
            return res;
        }

Here is the call graph for this function:

openTable()

Table::Table_Type icedb::Tables::CanHaveTables::openTable

(

const std::string &

tableName

)

Open the table with the matching name.

Exceptions

if	a table with this name does not exist.

Definition at line 321 of file Tables.cpp.

References icedb::Tables::Table::valid().

                                                                           {
            Expects(valid());
            const auto base = _getTablesParent();
            const auto tbl = std::make_shared<H5::DataSet>(base->openDataSet(tableName));
            Table::Table_Type res = std::make_unique<Table_impl>(tbl,tableName);
            return res;
        }

Here is the call graph for this function:

unlinkTable()

void icedb::Tables::CanHaveTables::unlinkTable

(

const std::string &

tableName

)

Unlink a table. In HDF5, this is not the same as erasing a table, which never actually happens.

Exceptions

if	a table with name tableName does not exist.
if	the open file is read only

Definition at line 315 of file Tables.cpp.

References icedb::Tables::Table::valid().

                                                                  {
            Expects(valid());
            auto base = _getTablesParent();
            base->unlink(tableName.c_str());
        }

Here is the call graph for this function:

valid()

bool icedb::Tables::CanHaveTables::valid ( ) const

private

Checks that the class is well-formed, constructed and activated. CanHaveTables needs post-constructor setup.

Definition at line 247 of file Tables.cpp.

{ if (_getTablesParent()) return true; return false; }

---

The documentation for this class was generated from the following files:

• lib/icedb/Table.hpp
• lib/src/Tables.cpp