Skip to main content
How are we doing? Please help us improve Stack Overflow. Take our short survey
358 questions
Newest Active Bountied Unanswered
Filter by
Sorted by
Tagged with
1 vote
0 answers
52 views

[Goal & Problem] I am trying to accelerate ONNX model inference on an RK3588 (Orange Pi 5) board using the Mali-G610 GPU. I have built OnnxRuntime (ORT) with the ACL (Compute Library) Execution ...
이호연's user avatar
  • 11
0 votes
0 answers
43 views

I created a YOLO v11 DL model using a given dataset and both the train and prediction modules work fine in PyCharm. However, after exporting the model file to ONNX format and running the prediction ...
Carlos Ni's user avatar
  • 1
0 votes
0 answers
36 views

I'm trying to bundle a Node.js CLI tool that uses @xenova/transformers into a single executable using pkg. The build works fine, but when I run the packaged executable, I get this error: Error: Cannot ...
Jibi Joseph's user avatar
  • 13
0 votes
0 answers
94 views

currently using both python and typescript for the motia based project (https://github.com/MotiaDev/motia), python mainly for working with onnxruntime-gpu. The OS environment is ubuntu 20.04. firstly ...
user824624's user avatar
  • 8,168
1 vote
0 answers
84 views

I am having trouble using onnxruntime.dll in ML.NET. My code works on one PC but not another. Clearly there is a mismatch between versions of some native DLLs, but I can't figure it out. The error is:...
Alex Gdalevich's user avatar
  • 711
0 votes
1 answer
165 views

I have a website that runs an onnx model (pre-optimized to .ort format) on the client via ONNX's web runtime. I must also stick to an old version of onnx (before 1.19.0), because the support for non-...
ph3rin's user avatar
  • 4,948
2 votes
1 answer
266 views

It is now not uncommon to have a python package that is distributed in a multitude of different "flavors". This happens often with machine learning packages, e.g. onnxruntime has many "...
MajorTom's user avatar
  • 385
0 votes
0 answers
36 views

Has anyone had any success running ort-web on a wasm backend with the proxy option (ort.env.wasm.proxy) set and multi-threading enabled? This is all the javascript I'm running: // alt.ts import * as ...
ack_inc's user avatar
  • 1,111
0 votes
2 answers
990 views

I'm following along with this YouTube video. I've already run the command pip install labelme opencv-python tensorflow matplotlib albumentations in the VS Code powershell terminal and successfully ...
Otakuwu's user avatar
  • 228
0 votes
0 answers
196 views

I am trying to use ONNX Runtime 1.22.0 which is in my application directory. I have added Include Directories, Library Directories, Linker, and command to copy dll to application directory under Pre-...
let me down slowly's user avatar
  • 1,027
0 votes
0 answers
88 views

My file structure to show what files are produced in dist when building the project: . ├── dist │ ├── 97d8ed7170daf34c90d5.wasm │ ├── content.js │ ├── content.js.LICENSE.txt │ ├── content.js....
miran80's user avatar
  • 1,135
1 vote
1 answer
475 views

I'm trying to extract a table from a jpeg image using Smoldocling. The result is great in 'doctags' format, but I can't export it : I don't have any warning nor error message, only an empty output. I ...
vvffl's user avatar
  • 105
0 votes
1 answer
3k views

I want to use Microsoft.ML.OnnxRuntime.Gpu version 1.20.1 on this tutorial on the ONNX website in dotnet 9.0.300. the tutorial uses CPU but I want to modify it to the GPU. However my modified code is ...
behroozbc's user avatar
  • 2,868
2 votes
3 answers
409 views

I need to use the onnxruntime with Java and spring boot to work with mkask detection model. but I have the below exception when calling OrtEnvironment env = OrtEnvironment.getEnvironment() I tried to ...
Rami Nassar's user avatar
  • 443
0 votes
0 answers
122 views

I'm currently working on running my custom ONNX segmentation model directly in the browser using ONNX Runtime for Web with HTML and JavaScript. The model is loading and running inference successfully. ...
Krishna Kurdekar's user avatar
  • 1
0 votes
1 answer
87 views

Running import onnxruntime as ort from datasets import load_dataset yields the error: (env-312) dernoncourt@pc:~/test$ python SE--test_importpb.py Traceback (most recent call last): File "/...
Franck Dernoncourt's user avatar
  • 84.7k
0 votes
0 answers
277 views

When I use silero-vad to reduce noise for my faster-whisper model, it reports the error below: --------------------------------------------------------------------------- ImportError ...
user22264203's user avatar
  • 11
0 votes
0 answers
91 views

Onnxruntime-inference-examples problem. I've been encountering issues with the multilingual Whisper-small model running on ONNX Runtime in my Android application. While the app launches successfully ...
Muh Sabry's user avatar
  • 11
0 votes
0 answers
25 views

I'm using InsightFace for face recognition on Windows with Python. When I initialize FaceAnalysis(), I get an error saying CUDAExecutionProvider couldn't be loaded, and it falls back to ...
Nick's user avatar
  • 66
0 votes
0 answers
255 views

I have found several documentation and tools for converting ONNX model to float16, but none of them supports converting to bfloat16. The model is originally trained using tensorflow and converted to ...
C Chen's user avatar
  • 1
0 votes
0 answers
84 views

I’m interested in using the TinyLlama-1.1B-Chat-v1.0 model with the react-native-transformers library on expo react native. The model works best with ONNX runtime and is more efficient on a Python ...
Lakpriya Senevirathna's user avatar
  • 5,369
0 votes
0 answers
272 views

I have trained a DL model on matlab and exported it as an ONNX file, now I need to run it on a raspberry pi 4 model B with Raspberry Pi OS X64 bit (aarch64) and python 3.11.2, I have followed many ...
Ahmad Omar's user avatar
  • 1
0 votes
0 answers
10 views

How to resolve this issue i've tried What went wrong: Could not determine the dependencies of task ':onnxruntime-react-native:compileDebugAidl'. > Could not resolve all task dependencies for ...
Mosi's user avatar
  • 1
0 votes
0 answers
44 views

I'm trying to make previsions with a ONNX model in a C# console application with ML.NET I'm getting the message: System.ArgumentException: 'Length of memory (10) must match product of dimensions (0).' ...
Codeman's user avatar
  • 1
1 vote
1 answer
204 views

I've built ONNX Runtime for Windows from scratch by cloning https://github.com/microsoft/onnxruntime and running the build.bat script. It successfully called down to CMake and compiled and generated a ...
Mark's user avatar
  • 5,543
0 votes
0 answers
89 views

In the onnxruntime documentation, for quantization here: https://onnxruntime.ai/docs/performance/model-optimizations/quantization.html#quantize-to-int4uint4 It sets accuracy_level=4 which means it's a ...
Owen Zhang's user avatar
  • 23
0 votes
2 answers
344 views

In my bitbake yocto development environment, I'm trying to get the onnxruntime library to not add this .debug folder to our development images as its too large for our devices we're building for. /usr/...
aoeu's user avatar
  • 154
2 votes
0 answers
606 views

I want to change onnx model and then restore the weights when using it with onnxruntime. However, the model doesn't seem to be changed. First, I load the existing model and change the weights based on ...
Eugene's user avatar
  • 255
0 votes
1 answer
77 views

I'm currently working on a .NET 8 project with the following setup: <Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> <OutputType>Exe</OutputType> ...
Le Magicien Annonyme's user avatar
  • 61
0 votes
1 answer
169 views

Thank you very much for reading my question , sorry if it is an obvious question. I use anaconda navigator : piped install the model whisper from OpenAi, which is an audio to text transformer model, I ...
Mickey Han's user avatar
  • 89
0 votes
0 answers
117 views

I'm currently running into an issue where running the changes (in green) in the following diff leads to worse performance on GPU (it adds an additional 100ms/request on average, with significantly ...
Bar-Levav's user avatar
  • 191
0 votes
1 answer
223 views

My model runs much slower in onnx than in torch. During the session initialization, I get some of these messages. [I:onnxruntime:, cuda_execution_provider.cc:2517 GetCapability] CUDA kernel not found ...
Mark's user avatar
  • 5,543
0 votes
0 answers
321 views

Using the yolov5 ultralytics repository, I trained a yolov5s model to detect certain objects using a dataset annotated in Roboflow with one object class. This model works great in a python repo when ...
Raiyan Rizwan's user avatar
  • 1
0 votes
0 answers
386 views

I am using ONNX to export a model trained in Python and run it in C#. Everything works rather well but I'd like to speed up the inference on the C# side, using Tasks or Threads on a CPU. I don't have ...
Christian's user avatar
  • 1,265
0 votes
0 answers
29 views

I want to implement the inference of onnx model in my own C code，but in some layers，the result between C and ONNX has 1 error, such as C is 40 but onnx is 41. I want to know why numpy's result is -87 ...
Chiourain Soong's user avatar
  • 1
1 vote
1 answer
74 views

My code uses PyTorch to perform segmentation annotations on PNG images. The input images have a width of 512 pixels or a multiple of this, but the height can range from 400 to 900 pixels. The code, ...
alanwilter's user avatar
  • 612
0 votes
1 answer
100 views

Here is myCMakeLists.txt cmake_minimum_required(VERSION 3.19) project(TesteVoxarApp LANGUAGES CXX) find_package(Qt6 6.5 REQUIRED COMPONENTS Core Widgets LinguistTools) find_path(...
rodrigo's user avatar
  • 68
1 vote
0 answers
336 views

I'm trying to deploy in production a LLM model with onnxruntime_go(https://github.com/yalue/onnxruntime_go). The problem is that I can not get the correctly answer, since I have loaded a tokenizer and ...
Wyatt's user avatar
  • 1,505
0 votes
0 answers
88 views

Referencing the onnxruntime documentation on custom operators ( https://onnxruntime.ai/docs/reference/operators/add-custom-op.html ), it's not clear to me whether: (A) Custom operators can be ...
Blunt Jackson's user avatar
  • 664
1 vote
2 answers
139 views

I have the following conanfile.py in my project on which I run the command conan install . --output-folder=build --build=missing -v. import os from conan import ConanFile class ProjectRecipe(...
Yaw Sparrow's user avatar
  • 65
-1 votes
1 answer
112 views

I am curious if anyone is doing this successfully, and if so how? I can build the shared libraries for inference successfully using the instructions on https://onnxruntime.ai/docs/build/inferencing....
Blunt Jackson's user avatar
  • 664
0 votes
0 answers
763 views

I'm encountering an issue when trying to run a model with ONNX Runtime using GPU acceleration on Windows. The error message indicates that the CUDAExecutionProvider cannot be loaded due to a ...
Sebastian H's user avatar
  • 131
1 vote
0 answers
214 views

My need is to use python and load/use LLAMA as LLM. I'm right now targeting llama2-7b. I'd like to ask which is the preferred way of executing such model on qcs 6490 using a python framework like. I ...
RJtokenring's user avatar
  • 497
0 votes
0 answers
93 views

I'm having a C# / .NET 4.7.2 project, running onnx runtime 1.10 for a long time. In the last year I started getting errors on some computers. I'm not sure where and why. Last week, it happens almost ...
ephraim's user avatar
  • 436
0 votes
1 answer
751 views

I’m exploring the idea of converting PyTorch and TensorFlow models to ONNX Runtime to optimize performance and reduce file sizes. Since most large language models (LLMs) are built using PyTorch, I was ...
Haseeb Sultan's user avatar
  • 103
2 votes
0 answers
354 views

I am new to Qualcomm's AI Engine Direct SDK (QNN). Before going directly to deploying AI models to Qualcomm's device, I would like to take a look at QNN's x86_64 backend, which is also relevant to QNN'...
fcc's user avatar
  • 187
1 vote
2 answers
507 views

I am trying to get my gpu to work with comfyui but I keep getting errors saying --- Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}} find model: E:\...
user26646123's user avatar
  • 11
0 votes
1 answer
135 views

I am trying to run yolov5 onnx model in Qualcomm NPU using OnnxRuntime QNN execution provider. I am getting this error while intializing the sesssion. ort.InferenceSession() ParseGraphInputOrOutput ...
Vignesh Chat's user avatar
  • 21
0 votes
1 answer
242 views

I use onnxruntime web with following code /** * * @param model don't pass session but pass model path and create session in infer inner. In this way, after infer finish, it will auto free gpu mem to ...
chikadance's user avatar
  • 4,195
1 vote
1 answer
4k views

After countless tries, I got confused in this issue but finally solved it. Now, back to the initial problem that made me lose my hair: CUDA does not seem to be used when I run my model with pytorch 2....
David Pagnon's user avatar
  • 113

1
2 3 4 5
…
8