Transitioners
Understanding and using transition models in HidTen.
Transitioners are components that compute transition probabilities or scores between hidden states. They define how likely the model is to transition from one state to another.
Built-in Transitioner
Transition matrix
hidten.tf.transitioner.TFTransitioner provides basic transition matrix
functionality. The hidten.transitioner.Transitioner.allow property should be
used to restrict the state transition graph.
For a single-head HMM allow accepts a list of index pairs, describing directional edges between state pairs. For the multi-head case, we can allow like this:
from hidten.tf.transitioner import TFTransitioner
transitioner = TFTransitioner()
transitioner.hmm_config = HMMConfig(states=[5, 5])
transitioner.allow = [
(0, 0, 0), (0, 0, 3), (0, 1, 3), (0, 2, 1), (0, 2, 4),
(0, 3, 1), (0, 4, 1), (0, 4, 2), (0, 4, 3),
(1, 0, 2), (1, 1, 4), (1, 2, 4), (1, 3, 1), (1, 4, 1)
]
transitioner.initializer = [
0.4, 0.6, 1., 0.5, 0.5,
1., 0.1, 0.2, 0.7,
1., 1., 1., 1., 1.
]
transitioner.build()
print(transitioner.matrix())
tf.Tensor(
[[[0.39999998 0. 0. 0.6 0. ]
[0. 0. 0. 1. 0. ]
[0. 0.5 0. 0. 0.5 ]
[0. 1. 0. 0. 0. ]
[0. 0.09999999 0.19999999 0.7 0. ]]
[[0. 0. 1. 0. 0. ]
[0. 0. 0. 0. 1. ]
[0. 0. 0. 0. 1. ]
[0. 1. 0. 0. 0. ]
[0. 1. 0. 0. 0. ]]], shape=(2, 5, 5), dtype=float32)
The initializer is a list of probabilities that has the same length and order as used for allow.
Note: allow is a hard constraint. Disallowed transitions will not be taken into account during training or inference and they can never be non-zero.
Starting Distribution
The hidten.transitioner.Transitioner.allow_start property should be
used to restrict the starting distribution.
E.g. allowing only the first and third state as valid starting states would look like this:
from hidten.tf.transitioner import TFTransitioner
transitioner = TFTransitioner()
transitioner.hmm_config = HMMConfig(states=[5])
transitioner.allow_start = [0, 2]
transitioner.initializer_start = [0.4, 0.6]
transitioner.build()
print(transitioner.start_dist())
tf.Tensor([[0.39999998 0. 0.6 0. 0. ]], shape=(1, 5), dtype=float32)
Note: It is not always useful to restrict the starting distribution. You might just
want to provide a hidten.transitioner.Transitioner.initializer_start that
contains probabilities for all states:
from hidten.tf.transitioner import TFTransitioner
transitioner = TFTransitioner()
transitioner.hmm_config = HMMConfig(states=[5])
transitioner.initializer_start = [0.3, 0.5, 0., 0.1, 0.1]
transitioner.build()
print(transitioner.start_dist())
Note: In this example, the third state is initialized with a probability of zero, but since starting in this state is allowed (i.e. it has not been disallowed), a non-zero probability could be learned.
Creating Custom Transitioners
TODO: This section is not yet complete.
To create custom transitioners, inherit from hidten.tf.transitioner.TFTransitioner.