streamitでmachine translationのdemo

以下(model_streamit.py)を作成し、

from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer
import streamlit as st

@st.cache(allow_output_mutation=True)
def load_model_tokenizer():
    model = M2M100ForConditionalGeneration.from_pretrained("facebook/m2m100_418M")
    tokenizer = M2M100Tokenizer.from_pretrained("facebook/m2m100_418M", src_lang="ja", tgt_lang="en")
    
    return model, tokenizer

# adding the text that will show in the text box as default
default_value = "最後の晩餐に出席した。"

def main():
    model, tokenizer = load_model_tokenizer()
    sent = st.text_area("テキストを入力し、Ctrl+Enterで解析結果を表示します。", default_value, height = 275)
    max_length = st.sidebar.slider("Max Length", min_value = 10, max_value=100, value=50)

    encoded_ja = tokenizer(sent, return_tensors="pt")
    generated_tokens = model.generate(
        **encoded_ja,
        forced_bos_token_id=tokenizer.get_lang_id("en"),
        max_length=max_length,
    )
    generated_sequences = tokenizer.decode(generated_tokens[0], skip_special_tokens=True)

    st.write(generated_sequences)


if __name__ == "__main__":
    main()

実行。

streamlit run model_streamit.py

pythonの最初の環境構築メモ

conda init zsh
sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"


# conda create (GUIで操作しても良い)
conda create --name [環境名] python=3.7 --prefix=[prefix] -y
conda activate [環境名]
jupyter kernelspec list
ipython kernel install --user --name=[環境名] --display-name=[環境名]


setting.json(remote)
{
    "remote.autoForwardPortsSource": "output"
}

setting.json(workspace)
{
    "remote.autoForwardPortsSource": "output",
    "python.pythonPath": "/root/anaconda3/bin/python(pyenv path)"
}

# kedro new
kedro new
cd [project名]
git init
…
git push -u origin main


conda env update --file environment.yml
pip list --format freeze
name: [環境名]
channels:
  - conda-forge
  - anaconda
  - default
dependencies:
  - kedro
  - pre-commit
  - pytest
  - pytest-cov
  - tox
  - black
  - yapf
  - pylint
  - mypy
  - pip

絶対パスに変更(zsh) - reorder-python-imports **/*.py - absolufy-imports src/**/*.py --application-directories src/

git の subdirectoryをlocalにinstallする方法

stackoverflow.com

pytorch の referenceをinstallする方法を探してたら、これが便利そう

$ svn ls https://github.com/pytorch/vision.git/trunk/references/detection
README.md
coco_eval.py
coco_utils.py
engine.py
group_by_aspect_ratio.py
train.py
transforms.py
utils.py
$ svn ls https://github.com/pytorch/vision.git/trunk/references
classification/
detection/
segmentation/
similarity/
video_classification/
$ svn export https://github.com/pytorch/vision.git/trunk/references
A    references
A    references/classification
A    references/classification/README.md
A    references/classification/train.py
A    references/classification/train_quantization.py
...
  • utils の importができない場合のrename

stackoverflow.com

deep learning を mobileで実装するときのリンク集

エッジやmobileでdeep を使用するときに参考にするサイト。

github.com

hub.packtpub.com

www.mobindustry.net

https://heartbeat.fritz.ai/intro-to-machine-learning-on-android-how-to-convert-a-custom-model-to-tensorflow-lite-e07d2d9d50e3

https://heartbeat.fritz.ai/machine-learning-on-android-computer-vision-c38c4072acd8

www.slideshare.net

www.slideshare.net

www.slideshare.net

google colab で動画が再生できなかった。

import base64
import io

def play(file_path):
    video = io.open(file_path, 'r+b').read()
    encoded = base64.b64encode(video)
    return(HTML(data='''<video width="320" height="240" controls><source src="data:video/mp4;base64,{0}" type="video/mp4" /></video>'''.format(encoded.decode('ascii'))))

walkingmask.hatenablog.com