Examples

This page contains a few examples of how you can use HyFI-ML package.

Installation

You can install HyFI-ML using pip:

pip install hyfi-ml

Requirement already satisfied: hyfi-ml in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (0.2.0)
Requirement already satisfied: cleanlab<=2.7.0,>=2.0.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi-ml) (2.7.0)
Requirement already satisfied: click<9.0.0,>=8.1.3 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi-ml) (8.1.7)
Requirement already satisfied: evaluate<=0.4.3,>=0.4.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi-ml) (0.4.3)
Requirement already satisfied: hyfi<2.0.0,>=1.36.3 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi-ml) (1.36.4)
Requirement already satisfied: scikit-learn<=1.5.2,>=1.0.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi-ml) (1.5.2)
Requirement already satisfied: transformers<=4.45.1,>=4.42.3 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (4.45.1)
Requirement already satisfied: numpy~=1.22 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from cleanlab<=2.7.0,>=2.0.0->hyfi-ml) (1.26.4)
Requirement already satisfied: tqdm>=4.53.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from cleanlab<=2.7.0,>=2.0.0->hyfi-ml) (4.66.5)
Requirement already satisfied: pandas>=1.4.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from cleanlab<=2.7.0,>=2.0.0->hyfi-ml) (2.2.1)
Requirement already satisfied: termcolor>=2.4.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from cleanlab<=2.7.0,>=2.0.0->hyfi-ml) (2.5.0)

Requirement already satisfied: datasets>=2.0.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from evaluate<=0.4.3,>=0.4.0->hyfi-ml) (2.18.0)
Requirement already satisfied: dill in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from evaluate<=0.4.3,>=0.4.0->hyfi-ml) (0.3.8)
Requirement already satisfied: requests>=2.19.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from evaluate<=0.4.3,>=0.4.0->hyfi-ml) (2.32.3)
Requirement already satisfied: xxhash in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from evaluate<=0.4.3,>=0.4.0->hyfi-ml) (3.5.0)
Requirement already satisfied: multiprocess in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from evaluate<=0.4.3,>=0.4.0->hyfi-ml) (0.70.16)
Requirement already satisfied: fsspec>=2021.05.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from fsspec[http]>=2021.05.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (2024.2.0)
Requirement already satisfied: huggingface-hub>=0.7.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from evaluate<=0.4.3,>=0.4.0->hyfi-ml) (0.25.2)
Requirement already satisfied: packaging in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from evaluate<=0.4.3,>=0.4.0->hyfi-ml) (24.1)
Requirement already satisfied: chardet<=5.2.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (5.2.0)
Requirement already satisfied: colorama<0.5.0,>=0.4.3 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (0.4.6)
Requirement already satisfied: gdown<=5.1.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (5.1.0)
Requirement already satisfied: hydra-colorlog<2.0.0,>=1.2.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (1.2.0)
Requirement already satisfied: hydra-core<2.0.0,>=1.3.2 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (1.3.2)
Requirement already satisfied: joblib<=1.3.2,>=1.2.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (1.3.2)
Requirement already satisfied: matplotlib<=3.7.1 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (3.7.1)
Requirement already satisfied: pathspec>=0.9.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (0.12.1)
Requirement already satisfied: pyarrow<=15.0.2 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (15.0.2)
Requirement already satisfied: pydantic<3.0.0,>=2.3.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (2.9.2)
Requirement already satisfied: pydantic-settings<3.0.0,>=2.0.1 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (2.5.2)
Requirement already satisfied: python-dotenv<2.0.0,>=1.0.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (1.0.1)
Requirement already satisfied: scipy<=1.12.0,>1.10.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hyfi<2.0.0,>=1.36.3->hyfi-ml) (1.12.0)

Requirement already satisfied: threadpoolctl>=3.1.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from scikit-learn<=1.5.2,>=1.0.0->hyfi-ml) (3.5.0)

Requirement already satisfied: filelock in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from transformers<=4.45.1,>=4.42.3->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (3.16.1)
Requirement already satisfied: pyyaml>=5.1 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from transformers<=4.45.1,>=4.42.3->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (6.0.2)
Requirement already satisfied: regex!=2019.12.17 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from transformers<=4.45.1,>=4.42.3->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (2024.9.11)
Requirement already satisfied: safetensors>=0.4.1 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from transformers<=4.45.1,>=4.42.3->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (0.4.5)
Requirement already satisfied: tokenizers<0.21,>=0.20 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from transformers<=4.45.1,>=4.42.3->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (0.20.0)
Requirement already satisfied: accelerate>=0.26.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (1.0.0)
Requirement already satisfied: torch in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (2.4.1)
Requirement already satisfied: psutil in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from accelerate>=0.26.0->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (6.0.0)

Requirement already satisfied: pyarrow-hotfix in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from datasets>=2.0.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (0.6)
Requirement already satisfied: aiohttp in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from datasets>=2.0.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (3.10.9)
Requirement already satisfied: beautifulsoup4 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from gdown<=5.1.0->hyfi<2.0.0,>=1.36.3->hyfi-ml) (4.12.3)

Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from huggingface-hub>=0.7.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (4.12.2)
Requirement already satisfied: colorlog in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hydra-colorlog<2.0.0,>=1.2.0->hyfi<2.0.0,>=1.36.3->hyfi-ml) (6.8.2)
Requirement already satisfied: omegaconf<2.4,>=2.2 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hydra-core<2.0.0,>=1.3.2->hyfi<2.0.0,>=1.36.3->hyfi-ml) (2.3.0)
Requirement already satisfied: antlr4-python3-runtime==4.9.* in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from hydra-core<2.0.0,>=1.3.2->hyfi<2.0.0,>=1.36.3->hyfi-ml) (4.9.3)
Requirement already satisfied: contourpy>=1.0.1 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from matplotlib<=3.7.1->hyfi<2.0.0,>=1.36.3->hyfi-ml) (1.3.0)
Requirement already satisfied: cycler>=0.10 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from matplotlib<=3.7.1->hyfi<2.0.0,>=1.36.3->hyfi-ml) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from matplotlib<=3.7.1->hyfi<2.0.0,>=1.36.3->hyfi-ml) (4.54.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from matplotlib<=3.7.1->hyfi<2.0.0,>=1.36.3->hyfi-ml) (1.4.7)
Requirement already satisfied: pillow>=6.2.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from matplotlib<=3.7.1->hyfi<2.0.0,>=1.36.3->hyfi-ml) (10.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from matplotlib<=3.7.1->hyfi<2.0.0,>=1.36.3->hyfi-ml) (3.1.4)
Requirement already satisfied: python-dateutil>=2.7 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from matplotlib<=3.7.1->hyfi<2.0.0,>=1.36.3->hyfi-ml) (2.9.0.post0)
Requirement already satisfied: importlib-resources>=3.2.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from matplotlib<=3.7.1->hyfi<2.0.0,>=1.36.3->hyfi-ml) (6.4.5)

Requirement already satisfied: pytz>=2020.1 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from pandas>=1.4.0->cleanlab<=2.7.0,>=2.0.0->hyfi-ml) (2024.2)
Requirement already satisfied: tzdata>=2022.7 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from pandas>=1.4.0->cleanlab<=2.7.0,>=2.0.0->hyfi-ml) (2024.2)
Requirement already satisfied: annotated-types>=0.6.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from pydantic<3.0.0,>=2.3.0->hyfi<2.0.0,>=1.36.3->hyfi-ml) (0.7.0)
Requirement already satisfied: pydantic-core==2.23.4 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from pydantic<3.0.0,>=2.3.0->hyfi<2.0.0,>=1.36.3->hyfi-ml) (2.23.4)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from requests>=2.19.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (3.4.0)
Requirement already satisfied: idna<4,>=2.5 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from requests>=2.19.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from requests>=2.19.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (2.2.3)
Requirement already satisfied: certifi>=2017.4.17 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from requests>=2.19.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (2024.8.30)

Requirement already satisfied: sympy in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (1.13.3)
Requirement already satisfied: networkx in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (3.2.1)
Requirement already satisfied: jinja2 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (3.1.4)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (12.1.105)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (12.1.105)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (12.1.105)
Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (9.1.0.70)
Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (12.1.3.1)
Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (11.0.2.54)
Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (10.3.2.106)
Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (11.4.5.107)
Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (12.1.0.106)
Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (2.20.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (12.1.105)
Requirement already satisfied: triton==3.0.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (3.0.0)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (12.6.77)

Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (2.4.3)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (24.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (6.1.0)
Requirement already satisfied: yarl<2.0,>=1.12.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (1.14.0)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from aiohttp->datasets>=2.0.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (4.0.3)

Requirement already satisfied: zipp>=3.1.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from importlib-resources>=3.2.0->matplotlib<=3.7.1->hyfi<2.0.0,>=1.36.3->hyfi-ml) (3.20.2)
Requirement already satisfied: six>=1.5 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib<=3.7.1->hyfi<2.0.0,>=1.36.3->hyfi-ml) (1.16.0)
Requirement already satisfied: soupsieve>1.2 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from beautifulsoup4->gdown<=5.1.0->hyfi<2.0.0,>=1.36.3->hyfi-ml) (2.6)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from jinja2->torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (3.0.1)

Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from requests[socks]->gdown<=5.1.0->hyfi<2.0.0,>=1.36.3->hyfi-ml) (1.7.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from sympy->torch->transformers[torch]<=4.45.1,>=4.42.3->hyfi-ml) (1.3.0)
Requirement already satisfied: propcache>=0.2.0 in /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages (from yarl<2.0,>=1.12.0->aiohttp->datasets>=2.0.0->evaluate<=0.4.3,>=0.4.0->hyfi-ml) (0.2.0)

Note: you may need to restart the kernel to use updated packages.

Usage

from hyfiml import get_version

print(get_version())