%% if you are submitting an initial manuscript then you should have submission as an option here
%% if you are submitting a revised manuscript then you should have revision as an option here
%% otherwise options taken by the article class will be accepted
\documentclass[submission]{FPSAC2019}
%% but DO NOT pass any options (or change anything else anywhere) which alters page size / layout / font size etc

\articlenumber{100}
\addbibresource{100.bib}

%% note that the class file already loads {amsmath, amsthm, amssymb}

\newtheorem{thm}{Theorem}
\newtheorem{lem}{Lemma}

\usepackage{lipsum}

%% define your title in the usual way
\title[Supporting Data-Driven Mathematics]{Supporting Data-Driven Mathematics: \\ Database and Interface Generation}

%% define your authors in the usual way
%% use \addressmark{1}, \addressmark{2} etc for the institutions, and use \thanks{} for contact details
\author{Katja Ber\v{c}i\v{c}\thanks{\href{mailto:katja.bercic@fau.de}{katja.bercic@fau.de} The author was supported by EU grant Horizon 2020 ERI 676541 OpenDreamKit}}

%% then use \addressmark to match authors to institutions here
\address{Department of Computer Science, FAU Erlangen-N\"urnberg, Germany}

%% put the date of submission here
\received{\today}

%% leave this blank until submitting a revised version
%\revised{}

%% put your English abstract here, or comment this out if you don't have one yet
%% please don't use custom commands in your abstract / resume, as these will be displayed online
%% likewise for citations -- please don't use \cite, and instead write out your citation as something like (author year)
\abstract{Easier access to computation is turning mathematics into an 
at least a partially data-driven discipline.
But it does not use the tools of the data trade:
a majority of datasets are still shared in an ad hoc manner.
We contribute an automated database setup process which builds a complete website stack,
from the database schema to a web interface, from a dataset declaration.}

%% put your French abstract here, or comment this out if you don't have one
%\resume{\lipsum[2]}

%% put your keywords here, or comment this out if you don't have them yet
\keywords{mathematical data, database framework, schema declaration}

%% you can include your bibliography however you want, but using an external .bib file is STRONGLY RECOMMENDED and will make the editor's life much easier
%% regardless of how you do it, please use numerical citations, ie. [xx, yy] in the text

%% this sample uses biblatex, which (among other things) takes care of URLs in a more flexible way than bibtex
%% but you can use bibtex if you want
%\usepackage[backend=bibtex]{biblatex}
%\addbibresource{bercic.bib}
%% note the \printbibliography command at the end of the file which goes with these biblatex commands

\DeclareMathOperator{\Tr}{Tr}
%\newcommand{\cn}[1]{\ensuremath{\mathtt{#1}}\xspace}
\newcommand{\chpoly}{\det(\lambda I - M)}

\usepackage{wrapfig}
\usepackage{xspace}
%MMT
\usepackage[dvipsnames]{xcolor}
\definecolor{backcolor}{gray}{.96}
\usepackage{listings}
\lstset{backgroundcolor=\color{backcolor},mathescape}
\input{listings-mmt}

\usepackage{xspace}
\newcommand{\mmt}{\texorpdfstring{{\normalfont\scshape{Mmt}}\xspace}{MMT\ }}
\newcommand\MDDL{\textsf{MDDL}\xspace}
\newcommand\MBGen{\textsf{MBGen}\xspace}
\newcommand\matrices{\textsf{Matrices}\xspace}
\newcommand\MDH{\textsf{MathDataHub}\xspace}

\begin{document}

\maketitle
%% note that you DO NOT have to put your abstract here -- it is generated by \maketitle and the \abstract and \resume commands above

\section{Introduction}
% a description of the context and state of the art

Generating data is becoming more and more common in mathematics,
but there are no easy ways to store and share it.
In particular, providing a user-friendly interface to a dataset
typically represents a non-trivial investment of resources for an author.
Many datasets, especially the smaller ones, end up stored and shared in an ad hoc manner.
Text files with one entry per line or 
code that produces an array of objects for a computer algebra system
(and similar) are not uncommon.

% In description: available math, data types - how extend (contact us) - future work: making it easier to extend

\paragraph{Related Work}
The work in progress living survey of mathematical databases~\cite{bercic:cmo:table}
is an indication of the status of data in mathematics.
It currently lists about a hundred datasets in roughly $50$ table entries.
The OEIS is the oldest mathematical database, and arguably the most influential one.
The LMFDB covers a wider scope and collects several datasets related via Langland's program~\cite{BerGel:ilp03}.
The OEIS and the LMFDB both have a substantial mathematical knowledge management architecture,
as do some other larger database projects.
Several datasets have also been integrated with computer algebra systems such as GAP, Magma or SageMath.
Both of these options are out of reach for smaller collaborations.

Some projects, like the LMFDB and the House of Graphs, accept dataset submissions.
However, they are all limited to a specific area of mathematics,
and the authors need to be registered users.
Generic research data hosting services are not extensively used by mathematicians,
likely due to the combination of the lack of awareness of them and the lack of added value they provide.
Some such are the EU infrastructures EUDAT and EOSC, RADAR, and hosting provided by publishers.
Tools for managing and browsing general databases are similarly unused.
They tend to be rich in features that are not useful from the perspective of mathematical datasets 
and have an initially steep learning curve.
Somewhat more surprising is that even simpler database tools, 
e.g. those based on the light-weight SQLite database,
appear to not be used much or at all.

The framework presented in this abstract is called \MBGen~\cite{MBGen:on}
and it builds on two projects.
The concepts of mathematical schemata and codecs were
developed during the OpenDreamKit project~\cite{WieKohRab:vtuimkb17}.
The interface that is produced is essentially an instance of a DiscreteZOO website~\cite{DZWeb:on},
which is a part of a larger project that also includes a data repository and a SageMath package.
The DiscreteZOO project~\cite{DZ:ICMS} was developed for the use case of collections of graphs with a high degree of symmetry,
but was designed to be useful in a more general setting.

\paragraph{Contribution}
We automate the process of setting up a database with a web interface for mathematical datasets.
\MBGen takes a dataset declaration as input and generates 
a complete website stack, from the database schema to a web interface.
The contribution is twofold and in the original spirit of DiscreteZOO.
Authors of mathematical datasets will benefit from an automated process of database setup.
In turn, this should lead to more datasets being available with a GUI and easy to use search features, 
which will benefit the researchers (data consumers and users).
A standardized interface can also form a basis for cross-database sharing and linking in the future.
In most cases, even a simple software stack would be a great improvement over the form a dataset is currently shared as.

\paragraph{Running example}
The dataset shown in \cref{tb:dataset} is simple on purpose and
we will use it to demonstrate the workings of \MBGen.
Mathilde has collected a set of integer matrices and computed 
their trace, eigenvalues, whether they are orthogonal, and their characteristic polynomial.
She wants to publish the dataset on her website.

\begin{table}\centering\def\arraystretch{1.5}
\begin{tabular}{ c c c l l }
  $M$ & $\Tr(M)$ & Orthogonal & $\sigma_M$ & $\chpoly$ \\\hline\hline
  $\bigl(\begin{smallmatrix}
  2 & 0 \\
  0 & 1
 \end{smallmatrix}\bigr)$ & $3$ & yes & $2$, $1$ & $\lambda^2 - 3\lambda + 2$ \\\hline
  $\bigl(\begin{smallmatrix}
  2 & 1 \\
  1 & 2
 \end{smallmatrix}\bigr)$ & $4$ & no & $3$, $1$ & $\lambda^2 - 4\lambda + 3$ \\\hline
  $\bigl(\begin{smallmatrix}
  -1 & 0 \\
  0 & 1
 \end{smallmatrix}\bigr)$ & $0$ & yes & $1$, $-1$& $\lambda^2 - 1$  \\\hline
\end{tabular}
\caption{Running example: Mathilde's matrix dataset}\label{tb:dataset}
\end{table}

\section{The \MBGen Framework}

% a description of the software itself: main functionalities , algorithms...
% a description of typical applications and use cases
% a comparison with existing software or algorithms
% brief evidence for the ease of use and reuse: documentation, examples of usage, availability, license, software requirements, installation instructions, (plans for) integration in existing platforms (e.g. GAP, Magma, Maple, Mathematica, or SageMath).
% brief evidence for software quality and sustainability: tests, development model, use of collaborative tools, etc.

\MBGen~\cite{MBGen:on} works for datasets that fit the following simple mold.
First, the dataset is composed of some number of lists of mathematical objects of the same type.
Second, for every type, the list consists of either simple objects or of object records.
These records could contain information about a set of mathematical properties or invariants,
provided they can be reasonably represented in a database.
The set of supported types is extensible and currently contains some basic types
(Booleans and integers) as well as some more complex types for demonstration.
An up-to-date description of the system, including the setup and 
a list of supported types is available in the project readme.

\paragraph{Writing a Dataset Declaration}
The schema declaration is written in the Mathematical Data Declaration Language (\MDDL)
and is essentially a list of declarations $c:A$,
where $c$ is a name of a property and $A$ is its type.
Every declaration also needs a codec, which is simply a pair of mappings
(an encoding and a decoding) between a mathematical type and a database type.
\MDDL and its underlying infrastructure were introduced and described in~\cite{CICM}.
For the purpose of this abstract, we only demonstrate how \MDDL can be used through examples,
and briefly describe codecs in the following paragraph.

Codecs are an essential part of the system that handles
the communication between the mathematical level and the database level.
They enable interoperability, which happens at the mathematical level,
and connect it with the storage at the database level.
Note that codecs can be arbitrary code and arbitrarily complex:
consider the example Brendan McKay's graph formats~\cite{GraphFormats},
which are widely used to represent graphs as strings.
These formats are typically used in combination with canonical labelings~\cite{Canonical}
to ensure that two graphs with the same encoding are necessarily also isomorphic.

To set up her project, Mathilde follows the steps described on the project page~\cite{MBGen:on}.
Once she is ready, she starts writing her schema declaration.
First, she just writes down an empty theory called \matrices (\hyperref[mmt:empty]{Listing~}\ref{mmt:empty}).
Next, she will write property declarations for each of the columns in her dataset,
as in \hyperref[mmt:property]{Listing~}\ref{mmt:property}.

\begin{minipage}[c]{7.5cm}
\begin{mmtcode}[caption={The empty theory},label=mmt:empty]
namespace http://data.mathhub.info/schemas❚
theory Matrices : ?MDDL =
❚
\end{mmtcode}
\end{minipage}\quad
\begin{minipage}[c]{7.5cm}
\begin{mmtcode}[caption={A property declaration},label=mmt:property]
property: propertyMathType ❘
    meta ?Codecs?codec propertyCodec ❘
    propertyTags ❙
\end{mmtcode}
\end{minipage}

She only needs to use types that are already supported, 
and for which codecs have already been defined.
In addition to the required information about the mathematical type
and the codecs, Mathilde can use the optional tags (\cref{tb:metadata})
to specify the interface behaviour.

\begin{table}
\begin{tabular}{ l l p{0.69\textwidth} }
\emph{Database} & \texttt{index} & Column representing a dataset index \\
\noalign{\vskip 1mm}\hline\noalign{\vskip 1mm}
\emph{Frontend} & \texttt{collection} & Collection metadata, e.g. provenance data \\
& \texttt{display} & Display name in the list of filters and in the results table head. \\
& \texttt{hidden} & By default, hide in the results display. \\
& \texttt{opaque} & Do not use for filtering. 
\end{tabular}
\caption{Metadata tags}\label{tb:metadata}
\end{table}

The tag \texttt{hidden} can be used when a dataset has many columns
(like the graph datasets in DiscreteZOO~\cite{DZWeb:on}).
If there are hidden columns, the interface shows an additional widget, 
which lets a visitor choose which columns they want to see.
If the tag \texttt{display} is not present, the column name is used for display.
The tags particularly relevant for the interface the metadata tags and the tags
\texttt{hidden} and \texttt{opaque}.

Now, Mathilde can write down her dataset declaration (\hyperref[mmt:schema]{Listing~}\ref{mmt:schema}).

\begin{mmtcode}[caption={Mathilde's dataset declaration},label=mmt:schema]
theory Matrices : ?MDDL =
meta /schemas?MDDL?datasetName "Mathilde's matrix dataset" ❙
    mat: matrix $\mathbb{Z}$ 2 2 ❘ meta ?Codecs?codec MatrixAsArray IntIdent ❘
        tag ?MDDL?opaque ❙
    trace: $\mathbb{Z}$ ❘ meta ?Codecs?codec IntIdent ❙
    orthogonal: bool ❘ meta ?Codecs?codec BoolIdent ❙
    eigenvalues: list $\mathbb{Z}$ ❘ meta ?Codecs?codec ListAsArray IntIdent ❘
        tag ?MDDL?opaque ❙
    characteristic: Polynomial IntegerRing ❘ meta ?Codecs?codec PolynomialAsSparseArray IntIdent ❘
        tag ?MDDL?opaque ❙   
❚
\end{mmtcode}

\medskip\noindent\begin{minipage}[c]{10.8cm}
\paragraph{Generating and Populating the Database}
For every schema theory $T$, \MBGen  generates an SQL table.
The table name is the theory name and for each declaration $c:A$ in $T$,
it generates a column, whose type is obtained from the codec declaration.
In addition to these, \MBGen generates a primary key column called \texttt{ID} of type \texttt{UUID}.
When Mathilde runs \MBGen on her schema theory from \hyperref[mmt:schema]{Listing~}\ref{mmt:schema}, 
she obtains the database schema on the right.
\end{minipage}\quad
\begin{minipage}[c]{5.2cm}
\begin{lstlisting}[basicstyle=\scriptsize\tt,columns=fixed,caption=Schema]
      Column     |   Type    
 ----------------+----------
  ID             | uuid      
  MAT            | integer[] 
  TRACE          | integer   
  ORTHOGONAL     | boolean   
  EIGENVALUES    | integer[] 
  CHARACTERISTIC | integer[] 
  
 Indexes: "Matrices_pkey" 
 PRIMARY KEY, btree ("ID")
\end{lstlisting}
\end{minipage}

If the dataset has a pre-existing ID-like field, which is not a \texttt{UUID},
the corresponding column should be tagged with \texttt{index}.
She can insert her data using the corresponding SQL \texttt{INSERT} statements, see \hyperref[tb:dbtable]{Listing~}\ref{tb:dbtable}.

\begin{lstlisting}[basicstyle=\scriptsize\tt,columns=fixed,caption=Table contents after insert,label=tb:dbtable]
         ID         |    MAT     | TRACE | ORTHOGONAL | EIGENVALUES | CHARACTERISTIC
 -------------------+------------+-------+------------+-------------+----------------
  e278b5e8-4404-... | {2,0,0,1}  |     3 | t          | {2,1}       | {0,2,1,-3,2,1}
  05a30ff0-4405-... | {2,1,1,2}  |     4 | f          | {3,1}       | {0,3,1,-4,2,1}
  1be3f022-4405-... | {-1,0,0,1} |     0 | t          | {1,-1}      | {0,-1,2,1}
\end{lstlisting}

\begin{figure}
\centering
\includegraphics[width=0.8\textwidth]{screenshot.png}
\caption{Screenshot of the website for Mathilde's use dataset}
\end{figure}

\paragraph{Extensions and Customisation}
Mathilde can modify the code and the database after they are generated 
and before running or deploying the website.
We use the project wiki~\cite{MBGen:on} to document how such modifications can be made.


\section{Conclusion and Future Work}
To evaluate the \MDH setup, we ran \MBGen for the datasets hosted on the DiscreteZOO website.
In some sense, the DiscreteZOO datasets are simpler than Mathilde's much smaller dataset, 
since they only contain Boolean and integer valued properties, while the objects (graphs) are string-encoded.
An author of a similarly uncomplicated dataset with no new codecs could start
from an existing dataset declaration and adapt it to their needs in under an hour.

\MBGen will be used as a critical first step in \MDH, a work in progress project
that has as its goal a unified mathematical data infrastructure, which will likely include hosting options.
As a part of that, we will continue to expand the core library of codecs and support for common data types.
Similarly, we intend to make it easier to add additional (custom) filters with future versions.
We will use~\cite{Rabe:qlfml12} as a starting point to explore possibilities 
for the development of mathematical query languages.
Finally, we aim to provide an API that would be suitable for providing access
from computational software such as SageMath or GAP.

\acknowledgements{
The author gratefully acknowledges Dennis M\"uller and Tom Wiesing's help in getting to grips with \mmt,
as well as Tom Wiesing's invaluable help in setting up the harmonious running of the various parts of the framework.
The author would also like to thank Michael Kohlhase for several constructive discussions which
were instrumental in setting the direction of the project and placing it into the larger setting.
The author is supported by the EU grant Horizon 2020 ERI 676541 OpenDreamKit.
}

%% if you use biblatex then this generates the bibliography
%% if you use some other method then remove this and do it your own way
\printbibliography



\end{document}
