自分で集めたデータでTensorFlow Lite for Microcontrollersのmagic wandの学習をしてみた

はじめに

環境

• PC
  • windows 10
• pico
  • https://www.switch-science.com/catalog/7412/
• USBケーブル
  • 家に転がってたものを
• 加速度センサー(MMA8452Q)
  • https://www.switch-science.com/catalog/1927/
  • ピンヘッダはあまりものを使ってはんだ付けした
• 330Ωの抵抗 x 2
  • 動作確認では手元になかったので使ってないが、ドキュメントを見た感じ合った方がよさそう。
• ブレッドボードやジャンパー
  • VDD/GND用 2 or 4本
  • SCL/SDA用 2本
• 開発環境
  • Visual Studio Code
  • Google Colabratory
• 動作確認環境
  • TeraTerm

流れ

1. データ収集
2. colabでの学習方法
3. Appendix: ローカル環境

1. データ収集

├─negative
│      output_negative_1.txt
│      output_negative_2.txt
│      output_negative_3.txt
│      output_negative_4.txt
│      output_negative_5.txt
│      output_negative_ccw_1.txt
│      output_negative_ccw_2.txt
│      output_negative_jiangyh.txt
│
├─ring
│      output_ring_01.txt
│      output_ring_02.txt
│      output_ring_03.txt
│      output_ring_04.txt
│      output_ring_05.txt
│      output_ring_06.txt
│      output_ring_07.txt
│      output_ring_08.txt
│      output_ring_09.txt
│
├─slope
│      output_slope_01.txt
│      output_slope_02.txt
│      output_slope_03.txt
│      output_slope_04.txt
│      output_slope_05.txt
│      output_slope_06.txt
│      output_slope_07.txt
│      output_slope_08.txt
│      output_slope_09.txt
│
└─wing
        output_wing_01.txt
        output_wing_02.txt
        output_wing_03.txt
        output_wing_04.txt
        output_wing_05.txt
        output_wing_06.txt
        output_wing_07.txt
        output_wing_08.txt
        output_wing_09.txt

2. colabでの学習

変更点1. tensorflowのversion

• (2022/05/28)のcolabだとtensorflowが2.8.0
• exampleのREADME だと tensorflow 2.0
• exampleのrequirements.txtだと tensorflow 2.5.3
• Interface 2021年8月号 ではtensorflow 2.1

tensorflow 2.8.0

"Invoke() called after initialization failed\n"

tensorflow 2.5.3

2022-05-28 23:41:08.599152: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
2022-05-28 23:41:08.599522: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 2199995000 Hz
Epoch 1/50
2022-05-28 23:41:09.456942: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2022-05-28 23:41:10.936803: E tensorflow/stream_executor/cuda/cuda_dnn.cc:352] Loaded runtime CuDNN library: 8.0.5 but source was compiled with: 8.1.0.  CuDNN library needs to have matching major version and equal or higher minor version. If using a binary install, upgrade your CuDNN library.  If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.
2022-05-28 23:41:10.939875: E tensorflow/stream_executor/cuda/cuda_dnn.cc:352] Loaded runtime CuDNN library: 8.0.5 but source was compiled with: 8.1.0.  CuDNN library needs to have matching major version and equal or higher minor version. If using a binary install, upgrade your CuDNN library.  If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.
Traceback (most recent call last):
  File "train.py", line 198, in <module>
    test_len, test_data, args.model)
  File "train.py", line 145, in train_net
    callbacks=[tensorboard_callback])
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py", line 1178, in fit
    tmp_logs = self.train_function(iterator)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 889, in __call__
    result = self._call(*args, **kwds)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 950, in _call
    return self._stateless_fn(*args, **kwds)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 3024, in __call__
    filtered_flat_args, captured_inputs=graph_function.captured_inputs)  # pylint: disable=protected-access
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 1961, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 596, in call
    ctx=ctx)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.UnknownError:  Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
	 [[node sequential/conv2d/Conv2D (defined at train.py:145) ]] [Op:__inference_train_function_1632]

Function call stack:
train_function

tensorflow 2.1

Built CNN.
Traceback (most recent call last):
  File "train.py", line 194, in <module>
    model, model_path = build_net(args, seq_length)
  File "train.py", line 103, in build_net
    model, model_path = build_cnn(seq_length)
  File "train.py", line 72, in build_cnn
    model.load_weights("./netmodels/CNN/weights.h5")
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/keras/engine/training.py", line 234, in load_weights
    return super(Model, self).load_weights(filepath, by_name, skip_mismatch)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/keras/engine/network.py", line 1222, in load_weights
    hdf5_format.load_weights_from_hdf5_group(f, self.layers)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/keras/saving/hdf5_format.py", line 651, in load_weights_from_hdf5_group
    original_keras_version = f.attrs['keras_version'].decode('utf8')
AttributeError: 'str' object has no attribute 'decode'

--- a/tensorflow/lite/micro/examples/magic_wand/train/train.py
+++ b/tensorflow/lite/micro/examples/magic_wand/train/train.py
@@ -69,7 +69,7 @@ def build_cnn(seq_length):
   print("Built CNN.")
   if not os.path.exists(model_path):
     os.makedirs(model_path)
-  model.load_weights("./netmodels/CNN/weights.h5")
+  #model.load_weights("./netmodels/CNN/weights.h5")
   return model, model_path

変更点2. data_prepare.py

--- a/tensorflow/lite/micro/examples/magic_wand/train/data_prepare.py
+++ b/tensorflow/lite/micro/examples/magic_wand/train/data_prepare.py
@@ -36,15 +36,14 @@ LABEL_NAME = "gesture"
 DATA_NAME = "accel_ms2_xyz"
 folders = ["wing", "ring", "slope"]
 names = [
-    "hyw", "shiyun", "tangsy", "dengyl", "zhangxy", "pengxl", "liucx",
-    "jiangyh", "xunkai"
+ "01", "02","03","04","05","06","07","08","09"
 ]

変更点3. data_split_person.py

--- a/tensorflow/lite/micro/examples/magic_wand/train/data_split_person.py
+++ b/tensorflow/lite/micro/examples/magic_wand/train/data_split_person.py
@@ -59,11 +59,11 @@ def person_split(whole_data, train_names, valid_names, test_names):  # pylint: d
 if __name__ == "__main__":
   data = read_data("./data/complete_data")
   train_names = [
-      "hyw", "shiyun", "tangsy", "dengyl", "jiangyh", "xunkai", "negative3",
+      "01", "02", "03", "04", "05", "negative3",
       "negative4", "negative5", "negative6"
   ]
-  valid_names = ["lsj", "pengxl", "negative2", "negative7"]
-  test_names = ["liucx", "zhangxy", "negative1", "negative8"]
+  valid_names = ["06", "08", "negative2", "negative7"]
+  test_names = ["07", "09","negative1", "negative8"]
   train_data, valid_data, test_data = person_split(data, train_names,
                                                    valid_names, test_names)
   if not os.path.exists("./person_split"):

変更点4: data_augmentation.py

例外が発生しました: IndexError
list index out of range
  File "D:\code\micon\tflite\tflite_micro\tensorflow\lite\micro\examples\magic_wand\train\data_load.py", line 69, in pad
    tmp_data = (np.random.rand(seq_length, dim) - 0.5) * noise_level + data[0]
  File "D:\code\micon\tflite\tflite_micro\tensorflow\lite\micro\examples\magic_wand\train\data_load.py", line 87, in format_support_func
    padded_data = self.pad(data, self.seq_length, self.dim)
  File "D:\code\micon\tflite\tflite_micro\tensorflow\lite\micro\examples\magic_wand\train\data_load.py", line 99, in format
    self.train_len, self.train_data = self.format_support_func(
  File "D:\code\micon\tflite\tflite_micro\tensorflow\lite\micro\examples\magic_wand\train\train.py", line 96, in load_data
    data_loader.format()
  File "D:\code\micon\tflite\tflite_micro\tensorflow\lite\micro\examples\magic_wand\train\train.py", line 187, in <module>
    load_data("./person_split/train", "./person_split/valid",

if data_type == "train":
      data, label = augment_data(data, label)

--- a/tensorflow/lite/micro/examples/magic_wand/train/data_augmentation.py
+++ b/tensorflow/lite/micro/examples/magic_wand/train/data_augmentation.py
@@ -61,7 +61,10 @@ def augment_data(original_data, original_label):
     # Time warping
     fractions = [(3, 2), (5, 3), (2, 3), (3, 4), (9, 5), (6, 5), (4, 5)]
     for molecule, denominator in fractions:
-      new_data.append(time_wrapping(molecule, denominator, data))
+      d = time_wrapping(molecule, denominator, data)
+      if len(d) == 0:
+        continue
+      new_data.append(d)
       new_label.append(label)

終わりに

3. Appendix: ローカル環境

python環境の構築

Python 3.6～3.9 Python 3.9 サポートには、TensorFlow 2.5 以降が必要です。 Python 3.8 サポートには、TensorFlow 2.2 以降が必要です。

venv環境構築

$ cd {path/to/train}
$ python -m venv .venv
$ .\venv\Scripts\activate
$ pip install --upgrade pip
$ pip install tensorflow==2.1

VS Codeでデバッガに引数を渡す

{
    // IntelliSense を使用して利用可能な属性を学べます。
    // 既存の属性の説明をホバーして表示します。
    // 詳細情報は次を確認してください: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "train",
            "type": "python",
            "request": "launch",
            "program": "train.py",
            "console": "integratedTerminal",
            "justMyCode": true,
            "args": ["--model", "CNN", "--person", "true"]
        }
    ]
}

ちなみに実行は左側にある「実行とデバッグ」(三角形と虫のアイコン) を押して、 launch.jsonで指定したname: trainの横にある三角ボタンを押すと実行でき、ブレイクポイントなどデバッグできるようになる。

VSCodeでファイルを開いているとそのウィンドウの右上に再生ボタンが出てくるが、そっちだとlaunch.jsonに紐づかないみたいなので、引数がない単体のpythonを実行したい時だけ使える。

Windowsでxxdコマンド