We will be working through a full-fledged tutorial in this tutorial. Workflow for Qualcomm AI Hub Models. We first set up the package required, discover the model collection available, and load MobileNet-V2 into PyTorch for local inference. Also, we handle an input-shape problem by converting NHWC tensors to the NCHW formats expected by the model. We then run inferences on the model’s sample input, as well the real image. Next, we inspect the top predictions. Finally, the Qualcomm AI Hub CLI is executed. And finally, the workflow extends with a YOLOv7 Object Detection example. We also include an optional section for cloud devices where we profile and run the model using a real Qualcomm Device when API tokens are available.
import subprocess, sys, os, glob, textwrap, traceback
Import numpy as torch, np
Import PIL Image
Matplotlib.pyplot can be imported as a plt
def pip_install(*pkgs):
subprocess.run([sys.executable, "-m", "pip", "install", "-q", *pkgs], check=True)
pip_install("qai_hub_models")
OUT_DIR = "/content/qaihm_out"; os.makedirs(OUT_DIR, exist_ok=True)
torch.set_grad_enabled(False)
Def to_nchw (value)
Arr = Value[0] If value is not in the list (tuple, list), then it's in the other case.
t = torch.from_numpy(np.asarray(arr, dtype=np.float32))
if t.ndim = 3
Unsqueeze t.
If t.ndim >= 4 and T.shape[1] The shape of t.shape is a 3[-1] == 3:
Contiguous()
The return of the t
To begin, we import libraries and set up the helper function that will install packages inside Colab. In order to only use inference, we disable gradient tracking, install qai_hub_models (and create an output directory). The to_nchw parameter is defined.() Function to convert input images tensors to the format required by the model.
import pkgutil, qai_hub_models.models as _m
model_ids = sorted(n for _, n, p in pkgutil.iter_modules(_m.__path__)
If (p) and n.startswith."_"))
print(f">>> {len(model_ids)} models available. First 40:n")
print(textwrap.fill(", ".join(model_ids[:40]), 100), "n")
from qai_hub_models.models.mobilenet_v2 import Model as MobileNetV2
model = MobileNetV2.from_pretrained().eval()
spec = model.get_input_spec()
input_name = list(spec.keys())[0]
print(">>> Input:"The input_name is spec[input_name]The shape, the specification[input_name].dtype)
Import MobileNet_V2_Weights from torchvision.models
IMAGENET_CLASSES = MobileNet_V2_Weights.IMAGENET1K_V1.meta["categories"]
Def Top5(logits:
If logits.ndim >= 1: Logits = logits.unsqueeze(0)
probs = torch.softmax(logits, dim=1)[0]
If you are unsure of the conf.topk(5), please contact us.
Return to the Homepage [(IMAGENET_CLASSES[i], float(c)) for c, i in zip(conf, idx)]
To understand the models available, we print out the first model IDs. After loading the MobileNet V2 model we read its specification and determine the correct input. The ImageNet class labeling is also prepared.() Convert model logits to readable Top-5 Predictions.
sample = model.sample_inputs()
x = to_nchw(sample[input_name])
print(">>> fed tensor shape:", tuple(x.shape))
print("n>>> Top-5 for the built-in sample input:")
For label, conf in top5(model (x))
print(f" {conf:6.2%} {label}")
Torchvision transforms
Preprocess = compose([
transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor(),
])
Img = None
try:
Import urllib.request
p = os.path.join(OUT_DIR, "input.jpg")
urllib.request.urlretrieve(
"https://raw.githubusercontent.com/pytorch/hub/master/images/dog.jpg", p)
img = Image.open(p).convert("RGB")
Except as follows:
print(">>> photo download skipped:", e)
If img is None, then:
preds = top5(model(preprocess(img).unsqueeze(0)))
print("n>>> Top-5 for the downloaded photo:")
Label, print in the preds (f" {conf:6.2%} {label}")
plt.figure(figsize=(5,5)); plt.imshow(img); plt.axis("off")
plt.title(f"{preds[0][0]} ({preds[0][1]:.1%})"); plt.show()
First, we run an inference by using a sample input that is built into the model and then use to_nchw() Fix the shape of the tensor before sending it to MobileNet V2. After downloading a real photo, we preprocess the image using standard resizing steps and conversion to tensors. Finally, we run another forecast. The top label of the predicted image is displayed to help visually link the output model to the original photo.
Def run_demo (module, extra=None and timeout=900).
After a cmd, you will see a =. [sys.executable, "-m", module, "--eval-mode", "fp",
"--output-dir", OUT_DIR] + (extra or [])
print(f"n>>> {' '.join(cmd)}")
try:
R = subprocess.run (cmd; capture_output=True; text=True; timeout=timeout);
print("n".join((r.stdout + r.stderr).strip().splitlines()[-25:]))
Exception to the rule:
print(">>> demo skipped:", e)
run_demo("qai_hub_models.models.mobilenet_v2.demo")
try:
pip_install("qai_hub_models[yolov7]")
run_demo("qai_hub_models.models.yolov7.demo")
imgs = sorted(glob.glob(OUT_DIR + "/*.png") + glob.glob(OUT_DIR + "/*.jpg"),
key=os.path.getmtime)
If imgs
plt.figure(figsize=(9,9)); plt.imshow(Image.open(imgs[-1]).convert("RGB"))
plt.axis("off"); plt.title("YOLOv7 detections"); plt.show()
else:
print(">>> no output image found (results may have printed instead).")
The exception:
print(">>> YOLOv7 section skipped:n", traceback.format_exc())
We define a reusable run_demo() This command executes the official Qualcomm AI hub model demos. It is used to install the YOLOv7 object detection extras and run the MobileNet V2 demo. The YOLOv7 Demo is run, we search for an output image and then visualize detections if one has been created.
try:
Importing qai_hub to hub
devices = hub.get_devices()
print(f"n>>> Authenticated. {len(devices)} cloud devices available.")
device = hub.Device("Samsung Galaxy S24 (Family)")
sample = model.sample_inputs()
nchw = to_nchw(sample[input_name])
traced = torch.jit.trace(model, [nchw])
cloud_inputs = {input_name: [nchw.numpy()]}
cj = hub.submit_compile_job(model=traced, device=device,
input_specs=model.get_input_spec(),
options="--target_runtime tflite")
target = cj.get_target_model(); print(">>> compiled:", cj.url)
pj = hub.submit_profile_job(model=target, device=device); print(">>> profiling:", pj.url)
ij = hub.submit_inference_job(model=target, device=device, inputs=cloud_inputs)
out = ij.download_output_data()
dev_logits = torch.from_numpy(np.asarray(list(out.values())[0][0]))
print(">>> Top-5 from the REAL device:")
Print(f) for Label, Conf in Top5(dev_logits)." {conf:6.2%} {label}")
target.download(os.path.join(OUT_DIR, "mobilenet_v2.tflite"))
print(">>> saved compiled .tflite to", OUT_DIR)
However, e.g.
print("n>>> Cloud (on-device) section skipped — no API token configured.")
print(" Get one at workbench.aihub.qualcomm.com, then:")
print(" !qai-hub configure --api_token YOUR_TOKEN")
print(" detail:", (str(e).splitlines() The following are some examples of how to use [type(e).__name__])[0])
print("n>>> Tutorial complete. Outputs in:", OUT_DIR)
A Qualcomm AI Hub optional cloud workflow is included. This runs when the API token has been configured. Retrieve cloud devices. Trace the PyTorch Model, compile for TFLite. Profile it on a Qualcomm Device. Submit an inference Job. Then we download the output from the device, print out the best predictions, then save the compiled TFLite models. Finally, we show where the tutorial outputs can be found.
As a conclusion, you now have an entire practical workflow that enables the use of Qualcomm AI Hub Models within Colab. The official demonstrations were used as reproducible references. We learned to load pre-trained models, correctly prepare inputs, perform local inferences, visually analyze classification and detection results and run local inferences. We saw that the model was able to be moved beyond PyTorch’s local execution and into Qualcomm’s cloud-device pipe-line for compilation, profile, and real device inference. This provides a way to move from simple experiments into hardware-aware implementation with Qualcomm AI hub.
Check out the Full Codes with Notebook here. Also, feel free to follow us on Twitter Don’t forget about our 150k+ ML SubReddit Subscribe Now our Newsletter. Wait! Are you using Telegram? now you can join us on telegram as well.
You can partner with us to promote your GitHub Repository OR Hugging Page OR New Product Launch OR Webinar, etc.? Connect with us
Post A Hands-On Coding Tutorial on Qualcomm AI Hub Models for Classification, Object Detection, and Hardware-Aware Deployment First on MarkTechPost.

