Heya

I stumbled across this project looking for some PyCUDA routines that operate on matrices per-row or per-column. It seems you have a bunch of handy routines for this, which is awesome, e.g. row-wise maximum, add_vec_to_mat etc.

Would you be willing to contribute them back to PyCUDA? a lot of these routines seem like they'd definitely be useful more widely. And perhaps offering the contribution might give the PyCUDA guys some inspiration or a kick in the arse to create a more general partial reductions API (like numpy's axis=0 arguments) and broadcasting behaviour for element-wise operations on GPUArrays? (I would attempt this myself but my CUDA-fu is weak)

Just a thought anyway. I would suggest it to them myself but the licencing is different (GPL vs MIT)

Cheers!

I am merging my code for training conv-nets from DNA sequence into Hebel. This should be done by the end of January 2015. Please follow this issue if you are interested in using Hebel for learning from DNA sequence or would like to test it.

I am trying to compile in Mac OSX yosemite and it seems hebel is not running. i installed PyCUDA and other libraries needed but stuck at this error.

$ python hebel_test.py Traceback (most recent call last): File "hebel_test.py", line 18, in hebel.init(0) File "/Users/prabhubalakrishnan/Desktop/hebel/hebel/init.py", line 131, in init from pycuda import gpuarray, driver, curandom File "/Library/Python/2.7/site-packages/pycuda-2014.1-py2.7-macosx-10.10-intel.egg/pycuda/gpuarray.py", line 3, in import pycuda.elementwise as elementwise File "/Library/Python/2.7/site-packages/pycuda-2014.1-py2.7-macosx-10.10-intel.egg/pycuda/elementwise.py", line 34, in from pytools import memoize_method File "/Library/Python/2.7/site-packages/pytools-2014.3.5-py2.7.egg/pytools/init.py", line 5, in from six.moves import range, zip, intern, input ImportError: cannot import name intern

How to fix?

When running optimizer.run(100), an error occurred: global name 'hidden_inputs' is not defined in line 323 of ./hebel/hebel/models/neurals_net.py

Where to define the global variable 'hidden_inputs'? Thanks!

[email protected]:~/github/hebel$ echo $LD_LIBRARY_PATH /usr/local/cuda:/usr/local/cuda/bin:/usr/local/cuda/lib64:/home/ubgpu/torch/install/lib:/home/ubgpu/torch/install/lib [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ python train_model.py examples/mnist_neural_net_shallow.yml Traceback (most recent call last): File "train_model.py", line 39, in run_from_config(yaml_src) File "/home/ubgpu/github/hebel/hebel/config.py", line 41, in run_from_config config = load(yaml_src) File "/home/ubgpu/github/hebel/hebel/config.py", line 92, in load proxy_graph = yaml.load(string, **kwargs) File "/usr/local/lib/python2.7/dist-packages/yaml/init.py", line 71, in load return loader.get_single_data() File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 39, in get_single_data return self.construct_document(node) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 48, in construct_document for dummy in generator: File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 398, in construct_yaml_map value = self.construct_mapping(node) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208, in construct_mapping return BaseConstructor.construct_mapping(self, node, deep=deep) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133, in construct_mapping value = self.construct_object(value_node, deep=deep) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 90, in construct_object data = constructor(self, tag_suffix, node) File "/home/ubgpu/github/hebel/hebel/config.py", line 318, in multi_constructor mapping = loader.construct_mapping(node) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208, in construct_mapping return BaseConstructor.construct_mapping(self, node, deep=deep) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133, in construct_mapping value = self.construct_object(value_node, deep=deep) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 90, in construct_object data = constructor(self, tag_suffix, node) File "/home/ubgpu/github/hebel/hebel/config.py", line 318, in multi_constructor mapping = loader.construct_mapping(node) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208, in construct_mapping return BaseConstructor.construct_mapping(self, node, deep=deep) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133, in construct_mapping value = self.construct_object(value_node, deep=deep) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 90, in construct_object data = constructor(self, tag_suffix, node) File "/home/ubgpu/github/hebel/hebel/config.py", line 323, in multi_constructor classname = try_to_import(tag_suffix) File "/home/ubgpu/github/hebel/hebel/config.py", line 251, in try_to_import exec('import %s' % modulename) File "", line 1, in File "/home/ubgpu/github/hebel/hebel/layers/init.py", line 17, in from .dummy_layer import DummyLayer File "/home/ubgpu/github/hebel/hebel/layers/dummy_layer.py", line 17, in from .hidden_layer import HiddenLayer File "/home/ubgpu/github/hebel/hebel/layers/hidden_layer.py", line 25, in from ..pycuda_ops import linalg File "/home/ubgpu/github/hebel/hebel/pycuda_ops/linalg.py", line 32, in from . import cublas File "/home/ubgpu/github/hebel/hebel/pycuda_ops/cublas.py", line 47, in import cuda File "/home/ubgpu/github/hebel/hebel/pycuda_ops/cuda.py", line 36, in from cudadrv import * File "/home/ubgpu/github/hebel/hebel/pycuda_ops/cudadrv.py", line 233, in _libcuda.cuPointerGetAttribute.restype = int File "/usr/lib/python2.7/ctypes/init.py", line 378, in getattr func = self.getitem(name) File "/usr/lib/python2.7/ctypes/init.py", line 383, in getitem func = self._FuncPtr((name_or_ordinal, self)) AttributeError: python: undefined symbol: cuPointerGetAttribute [email protected]:~/github/hebel$

[email protected]:~/github/hebel$ sudo pip install pyCUDA Requirement already satisfied (use --upgrade to upgrade): pyCUDA in /usr/local/lib/python2.7/dist-packages Requirement already satisfied (use --upgrade to upgrade): decorator>=3.2.0 in /usr/local/lib/python2.7/dist-packages (from pyCUDA) Requirement already satisfied (use --upgrade to upgrade): pytools>=2011.2 in /usr/local/lib/python2.7/dist-packages (from pyCUDA) Requirement already satisfied (use --upgrade to upgrade): pytest>=2 in /usr/local/lib/python2.7/dist-packages (from pyCUDA) Requirement already satisfied (use --upgrade to upgrade): appdirs>=1.4.0 in /usr/local/lib/python2.7/dist-packages (from pytools>=2011.2->pyCUDA) Requirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python2.7/dist-packages (from pytools>=2011.2->pyCUDA) Requirement already satisfied (use --upgrade to upgrade): py>=1.4.25 in /usr/local/lib/python2.7/dist-packages (from pytest>=2->pyCUDA) [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ python Python 2.7.6 (default, Mar 22 2014, 22:59:56) [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information.

quit() [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ sudo pip install PyCUDA Requirement already satisfied (use --upgrade to upgrade): PyCUDA in /usr/local/lib/python2.7/dist-packages Requirement already satisfied (use --upgrade to upgrade): decorator>=3.2.0 in /usr/local/lib/python2.7/dist-packages (from PyCUDA) Requirement already satisfied (use --upgrade to upgrade): pytools>=2011.2 in /usr/local/lib/python2.7/dist-packages (from PyCUDA) Requirement already satisfied (use --upgrade to upgrade): pytest>=2 in /usr/local/lib/python2.7/dist-packages (from PyCUDA) Requirement already satisfied (use --upgrade to upgrade): appdirs>=1.4.0 in /usr/local/lib/python2.7/dist-packages (from pytools>=2011.2->PyCUDA) Requirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python2.7/dist-packages (from pytools>=2011.2->PyCUDA) Requirement already satisfied (use --upgrade to upgrade): py>=1.4.25 in /usr/local/lib/python2.7/dist-packages (from pytest>=2->PyCUDA) [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ echo $PYTHONPATH /usr/local/lib/python2.7/dist-packages [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ [email protected]:~/github/hebel$ python train_model.py examples/mnist_neural_net_shallow.yml Traceback (most recent call last): File "train_model.py", line 39, in run_from_config(yaml_src) File "/home/ubgpu/github/hebel/hebel/config.py", line 41, in run_from_config config = load(yaml_src) File "/home/ubgpu/github/hebel/hebel/config.py", line 92, in load proxy_graph = yaml.load(string, **kwargs) File "/usr/local/lib/python2.7/dist-packages/yaml/init.py", line 71, in load return loader.get_single_data() File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 39, in get_single_data return self.construct_document(node) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 48, in construct_document for dummy in generator: File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 398, in construct_yaml_map value = self.construct_mapping(node) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208, in construct_mapping return BaseConstructor.construct_mapping(self, node, deep=deep) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133, in construct_mapping value = self.construct_object(value_node, deep=deep) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 90, in construct_object data = constructor(self, tag_suffix, node) File "/home/ubgpu/github/hebel/hebel/config.py", line 318, in multi_constructor mapping = loader.construct_mapping(node) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208, in construct_mapping return BaseConstructor.construct_mapping(self, node, deep=deep) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133, in construct_mapping value = self.construct_object(value_node, deep=deep) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 90, in construct_object data = constructor(self, tag_suffix, node) File "/home/ubgpu/github/hebel/hebel/config.py", line 318, in multi_constructor mapping = loader.construct_mapping(node) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 208, in construct_mapping return BaseConstructor.construct_mapping(self, node, deep=deep) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 133, in construct_mapping value = self.construct_object(value_node, deep=deep) File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 90, in construct_object data = constructor(self, tag_suffix, node) File "/home/ubgpu/github/hebel/hebel/config.py", line 323, in multi_constructor classname = try_to_import(tag_suffix) File "/home/ubgpu/github/hebel/hebel/config.py", line 251, in try_to_import exec('import %s' % modulename) File "", line 1, in File "/home/ubgpu/github/hebel/hebel/layers/init.py", line 17, in from .dummy_layer import DummyLayer File "/home/ubgpu/github/hebel/hebel/layers/dummy_layer.py", line 17, in from .hidden_layer import HiddenLayer File "/home/ubgpu/github/hebel/hebel/layers/hidden_layer.py", line 25, in from ..pycuda_ops import linalg File "/home/ubgpu/github/hebel/hebel/pycuda_ops/linalg.py", line 32, in from . import cublas File "/home/ubgpu/github/hebel/hebel/pycuda_ops/cublas.py", line 47, in import cuda File "/home/ubgpu/github/hebel/hebel/pycuda_ops/cuda.py", line 35, in from cudart import * File "/home/ubgpu/github/hebel/hebel/pycuda_ops/cudart.py", line 60, in raise OSError('CUDA runtime library not found') OSError: CUDA runtime library not found

at commit a7f4cbb91c029c344921db76850bf9dc8eb47af4 with python 2.7.6 I try: 'python train_model.py examples/mnist_neural_net_shallow.yml' And i get the following

Traceback (most recent call last):
  File "train_model.py", line 39, in <module>
    run_from_config(yaml_src)
  File "/Users/epic/Documents/git/hebel/hebel/config.py", line 41, in run_from_config
    config = load(yaml_src)
  File "/Users/epic/Documents/git/hebel/hebel/config.py", line 92, in load
    proxy_graph = yaml.load(string, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/yaml/__init__.py", line 71, in load
    return loader.get_single_data()
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 39, in get_single_data
    return self.construct_document(node)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 48, in construct_document
    for dummy in generator:
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 398, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 208, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 133, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 90, in construct_object
    data = constructor(self, tag_suffix, node)
  File "/Users/epic/Documents/git/hebel/hebel/config.py", line 318, in multi_constructor
    mapping = loader.construct_mapping(node)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 208, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 133, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 90, in construct_object
    data = constructor(self, tag_suffix, node)
  File "/Users/epic/Documents/git/hebel/hebel/config.py", line 318, in multi_constructor
    mapping = loader.construct_mapping(node)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 208, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 133, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/usr/local/lib/python2.7/site-packages/yaml/constructor.py", line 90, in construct_object
    data = constructor(self, tag_suffix, node)
  File "/Users/epic/Documents/git/hebel/hebel/config.py", line 323, in multi_constructor
    classname = try_to_import(tag_suffix)
  File "/Users/epic/Documents/git/hebel/hebel/config.py", line 251, in try_to_import
    exec('import %s' % modulename)
  File "<string>", line 1, in <module>
  File "/Users/epic/Documents/git/hebel/hebel/layers/__init__.py", line 17, in <module>
    from .dummy_layer import DummyLayer
  File "/Users/epic/Documents/git/hebel/hebel/layers/dummy_layer.py", line 17, in <module>
    from .hidden_layer import HiddenLayer
  File "/Users/epic/Documents/git/hebel/hebel/layers/hidden_layer.py", line 25, in <module>
    from ..pycuda_ops import linalg
  File "/Users/epic/Documents/git/hebel/hebel/pycuda_ops/linalg.py", line 32, in <module>
    from . import cublas
  File "/Users/epic/Documents/git/hebel/hebel/pycuda_ops/cublas.py", line 47, in <module>
    import cuda
  File "/Users/epic/Documents/git/hebel/hebel/pycuda_ops/cuda.py", line 35, in <module>
    from cudart import *
  File "/Users/epic/Documents/git/hebel/hebel/pycuda_ops/cudart.py", line 142, in <module>
    _libcudart.cudaGetErrorString.restype = ctypes.c_char_p
AttributeError: 'NoneType' object has no attribute 'cudaGetErrorString'

Example script data_providers.py imports skdata by
from skdata.mnist.view import OfficialVectorClassification

It should be skdata.mnist.views, otherwise errors occur.

It's not enough to specify root package (i.e. hebel) in packages argument of setup.py. In order to successfully install hebel, subpackages must be listed as well.

There is a small typo in hebel/layers/hidden_layer.py, hebel/layers/linear_regression_layer.py, hebel/layers/logistic_layer.py, hebel/layers/softmax_layer.py.

Should read initialized rather than initalized.

Semi-automated pull request generated by https://github.com/timgates42/meticulous/blob/master/docs/NOTE.md

Hi there, I really appreciate Hebel. It was a good first step for me to "take the plunge" into using GPU.

I struggled a bit after going through the example (MNIST) script. In particular, it wasn't clear how to have the model predict new data (i.e., data you don't have targets for).

The first (small) stumble was what to with the DataProvider. I just put in dummy zero targets. Perhaps targets could be an optional field somehow?

A more thorny issue was how to actually do the predictions. I couldn't for the life of me figure out how to feed the DataProvider data into the feed_forward without getting the error:

  File "/usr/local/lib/python2.7/dist-packages/hebel/models/neural_net.py", line 422, in feed_forward
    prediction=prediction))
  File "/usr/local/lib/python2.7/dist-packages/hebel/layers/input_dropout.py", line 96, in feed_forward
    return (input_data * (1 - self.dropout_probability),)
TypeError: unsupported operand type(s) for *: 'MiniBatchDataProvider' and 'float'

This was my original attempt:

# After loading in the data . . .
Xv = Xv.astype(np.float32)
yv = pd.get_dummies(yv).values.astype(np.float32)
valid_data = MiniBatchDataProvider(Xv, yv, batch_size=5000)

I finally resorted to useing a gpu array which worked:

from pycuda import gpuarray
valid_data = gpuarray.to_gpu(Xt)
y_pred = model.feed_forward(valid_data, return_cache=False, prediction=True).get()

The .get() at the end of the last statement was also something I had to figure out going through code.

Having an example in the documentation would be helpful.