.. _sphx_glr_tutorials_models_4_old_wines:

.. _tutorials4-index:


Revisit classic models from a graph perspective
-------------------------------------------------------

* **Capsule** `[paper] <https://arxiv.org/abs/1710.09829>`__ `[tutorial]
  <4_old_wines/2_capsule.html>`__ `[PyTorch code]
  <https://github.com/dmlc/dgl/tree/master/examples/pytorch/capsule>`__:
  This new computer vision model has two key ideas. First, enhancing the feature
  representation in a vector form (instead of a scalar) called *capsule*. Second,
  replacing max-pooling with dynamic routing. The idea of dynamic routing is to
  integrate a lower level capsule to one or several higher level capsules
  with non-parametric message-passing. A tutorial shows how the latter can be 
  implemented with DGL APIs.


* **Transformer** `[paper] <https://arxiv.org/abs/1706.03762>`__ `[tutorial] <4_old_wines/7_transformer.html>`__ 
  `[PyTorch code] <https://github.com/dmlc/dgl/tree/master/examples/pytorch/transformer>`__ and **Universal Transformer** 
  `[paper] <https://arxiv.org/abs/1807.03819>`__ `[tutorial] <4_old_wines/7_transformer.html>`__
  `[PyTorch code] <https://github.com/dmlc/dgl/tree/master/examples/pytorch/transformer/modules/act.py>`__:
  These two models replace recurrent neural networks (RNNs) with several layers of multi-head attention to
  encode and discover structures among tokens of a sentence. These attention
  mechanisms are similarly formulated as graph operations with message-passing.



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Author: Jinjing Zhou, Jake Zhao, Zheng Zhang, Jinyang Li">

.. only:: html

  .. image:: /tutorials/models/4_old_wines/images/thumb/sphx_glr_2_capsule_thumb.png
    :alt:

  :ref:`sphx_glr_tutorials_models_4_old_wines_2_capsule.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Capsule Network</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Author: Zihao Ye, Jinjing Zhou, Qipeng Guo, Quan Gan, Zheng Zhang">

.. only:: html

  .. image:: /tutorials/models/4_old_wines/images/thumb/sphx_glr_7_transformer_thumb.png
    :alt:

  :ref:`sphx_glr_tutorials_models_4_old_wines_7_transformer.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Transformer as a Graph Neural Network</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /tutorials/models/4_old_wines/2_capsule
   /tutorials/models/4_old_wines/7_transformer