Structured State Space Models (SSMs) have traditionally excelled in continuous data modalities, leveraging their strong inductive bias, but struggle with discrete data such as text and DNA due to inherent limitations.
The employment of a selection mechanism within SSMs demonstrates a method to compress input data effectively, thus enhancing performance in discrete data contexts while still requiring careful consideration of trade-offs.
Collection
[
|
...
]