PyStruct aims at being an easy-to-use structured learning and prediction library. Currently it implements only max-margin methods and a perceptron, but other algorithms might follow.
The goal of PyStruct is to provide a well-documented tool for researchers as well as non-experts to make use of structured prediction algorithms. The design tries to stay as close as possible to the interface and conventions of scikit-learn.
Currently the project is mostly maintained by Andreas Mueller, but contributions are very welcome. I plan a stable release soon.
You can contact the authors either via the mailing list or on github.
There are three basic concepts in the implementation.
Know about learning.
These implement max margin learning, similar to SVM^struct. There is a subgradient and a QP version. It is possible to put positivity constraints on certain weight. There is also a simple perceptron.
Know about the model formulation.
These know about the structure of the model, the loss and the inference. This is basically the part that you have to write yourself when using the Python interface in SVM^struct. I am currently working only on pairwise models and there is support for grids and general graphs. The SSVM implementations are agnostic to the kind of model that is used, so you can easily extend the given models to include higher-order potentials, for example.
Do the inference.
There are some options to use different solvers for inference. A linear programming solver using GLPK is included. I have Python interfaces for several other methods on github, including LibDAI, QPBO, AD3 and GCO (submodular graph cuts).
This is where the heavy lifting is done and in some sense these backends are interchangeable.
Currently I would recommend AD3 for very accurate solutions and QPBO for larger models. The OneSlackSSVM includes an option (switch_to) to switch the solver to a stronger or exact solver when no constraints can be found using the previous solver (which should be a faster undergenerating solver, such as QPBO).
For updates, read my blog at http://peekaboo-vision.blogspot.com
There are not publications yet that you can cite for this, I’m hoping there will be some in the future.
Btw: this is research with unit tests!
There is no need to compile anything, this pure Python. (FIXME, Crammer-Singer has a cython part!)
There are quite a couple of requirements, though:
You need cvxopt for the cutting plane SVM solver and linear programming inference. By default I use the glpk solver for the LP, so you need that, too, if you want to use LP inference.
You need sklearn for some tidbits here and there, also I import joblib from sklearn.
For the other inference algorithms that are wrapped in the inference folder, you need the following of my repositories. You can just pick and choose from those, but lack of methods will make some tests fail.
QPBO https://github.com/amueller/pyqpbo
This module contains algorithms for solving the structured learning model. Most are based on structured support vector machines.
Currently, I advise to use the OneSlackSSVM, which solves the QP using CVXOPT. SubgradientSSVM is a very simple implementation, that also might be interesting.
NSlackSSVM is the n-slack formulation of the QP and should work reliably, but is not as optimized as OneSlackSSVM. The rest is experimental / for testing.
learners.OneSlackSSVM(model[, max_iter, C, ...]) | Structured SVM solver for the 1-slack QP with l1 slack penalty. |
learners.NSlackSSVM(model[, max_iter, C, ...]) | Structured SVM solver for the n-slack QP with l1 slack penalty. |
learners.SubgradientSSVM(model[, max_iter, ...]) | Structured SVM solver using subgradient descent. |
learners.StructuredPerceptron(model[, ...]) | Structured Perceptron training. |
learners.LatentSSVM(base_ssvm[, ...]) | Stuctured SVM solver for latent-variable models. |
learners.LatentSubgradientSSVM(model[, ...]) | Latent Variable Structured SVM solver using subgradient descent. |
learners.PrimalDSStructuredSVM(model[, ...]) | Uses downhill simplex for optimizing an unconstraint primal. |
This module contains model formulations for several settings. They provide the glue between the learning algorithm and the data (and inference). The BinarySVMModel implements a standard SVM, the CrammerSingerSVMModel a multi-class SVM - which is surprisingly efficient and sometimes comparable to LibLinear Crammer-Singer Implementation.
GraphCRF implements a simple pairwise model for arbitrary graphs, while EdgeFeatureGraphCRF allows for arbitrary features for each edge, symmetric, assymmetric and arbitrary potentials.
GridCRF is a convenience class for grid graphs.
models.BinarySVMModel(n_features) | Formulate standard linear binary SVM in CRF framework. |
models.CrammerSingerSVMModel(n_features[, ...]) | Formulate linear multiclass SVM in C-S style in CRF framework. |
models.GraphCRF([n_states, n_features, ...]) | Pairwise CRF on a general graph. |
models.EdgeFeatureGraphCRF([n_states, ...]) | Pairwise CRF with features/strength associated to each edge. |
models.LatentGraphCRF(n_labels[, ...]) | CRF with latent states for variables. |
models.LatentNodeCRF([n_labels, n_features, ...]) | CRF with latent variables. |
models.GridCRF([n_states, n_features, ...]) | Pairwise CRF on a 2d grid. |
models.DirectionalGridCRF([n_states, ...]) | CRF in which each direction of edges has their own set of parameters. |
inference.inference_dispatch(...[, relaxed, ...]) | Wrapper function to dispatch between inference method by string. |
inference.inference_qpbo(unary_potentials, ...) | Inference with PyQPBO backend. |
inference.inference_dai(unary_potentials, ...) | Inference with LibDAI backend. |
inference.inference_lp(unary_potentials, ...) | Inference with build-in LP solver using GLPK backend. |
inference.inference_ad3(unary_potentials, ...) | Inference with AD3 dual decomposition subgradient solver. |
utils.SaveLogger(file_name[, save_every, ...]) | Logging class that stores the model periodically. |
This module provides a callable for easy evaluation of stored models.
plot_learning.plot_learning(ssvm[, time]) | Plot optimization curves and cache hits. |