KServe Files

What's Changed

• bump to vllm0.6.2 and add explicit chat template by @hustxiayang in https://github.com/kserve/kserve/pull/3964
• bump to vllm0.6.3 by @hustxiayang in https://github.com/kserve/kserve/pull/4001
• Feature: Add hf transfer by @tjandy98 in https://github.com/kserve/kserve/pull/4000
• Fix snyk scan null error by @sivanantha321 in https://github.com/kserve/kserve/pull/3974
• Update quick install script by @johnugeorge in https://github.com/kserve/kserve/pull/4005
• Local Model Node CR by @HotsauceLee in https://github.com/kserve/kserve/pull/3978
• Reduce E2Es dependency on CI environment (2) by @israel-hdez in https://github.com/kserve/kserve/pull/4008
• Allow GCS to download single file by @spolti in https://github.com/kserve/kserve/pull/4015
• bump to vllm0.6.3.post1 by @hustxiayang in https://github.com/kserve/kserve/pull/4023
• Set default for SamplingParams.max_tokens in OpenAI requests if unset by @kevinmingtarja in https://github.com/kserve/kserve/pull/4020
• Add tools functionality to vLLM by @ArjunBhalla98 in https://github.com/kserve/kserve/pull/4033
• For vllm users, our parser should be able to support both - and _ by @hustxiayang in https://github.com/kserve/kserve/pull/3933
• Add tools unpacking for vLLM by @ArjunBhalla98 in https://github.com/kserve/kserve/pull/4035
• Multi-Node Inference Implementation by @Jooho in https://github.com/kserve/kserve/pull/3972
• Enhance InjectAgent to Handle Only HTTPGet, TCP Readiness Probes by @LOADBC in https://github.com/kserve/kserve/pull/4012
• Feat: Fix memory issue by replacing io.ReadAll with io.Copy (#4017) by @ops-jaeha in https://github.com/kserve/kserve/pull/4018
• Update alibiexplainer example by @spolti in https://github.com/kserve/kserve/pull/4004
• Fix huggingface build runs out of storage in CI by @sivanantha321 in https://github.com/kserve/kserve/pull/4044
• Update snyk scan to include new images by @sivanantha321 in https://github.com/kserve/kserve/pull/4042
• Introducing KServe Guru on Gurubase.io by @kursataktas in https://github.com/kserve/kserve/pull/4038
• Fix Hugging Face server EncoderModel not returning probabilities by correctly passing --return_probabilities flag (#3958) by @oplushappy in https://github.com/kserve/kserve/pull/4024
• Add deeper readiness check for transformer by @sivanantha321 in https://github.com/kserve/kserve/pull/3348
• Fix Starlette Denial of service (DoS) via multipart/form-data by @spolti in https://github.com/kserve/kserve/pull/4006
• remove duplicated import "github.com/onsi/gomega" by @carlory in https://github.com/kserve/kserve/pull/4051
• Fix localmodel controller name in snyk scan workflow by @sivanantha321 in https://github.com/kserve/kserve/pull/4054
• Fix azure blob storage access key env not mounted by @bentohset in https://github.com/kserve/kserve/pull/4064
• Storage Initializer support single digit azure DNS zone ID by @bentohset in https://github.com/kserve/kserve/pull/4070
• Fix trust remote code encoder model by @sivanantha321 in https://github.com/kserve/kserve/pull/4043
• introduce the prepare-for-release.sh script by @spolti in https://github.com/kserve/kserve/pull/3993
• Model cache controller and node agent by @yuzisun in https://github.com/kserve/kserve/pull/4089
• Storage containers typo fix for Huggingface Storage type by @andyi2it in https://github.com/kserve/kserve/pull/4098
• Support datetime object serialization in v1/v2 response by @sivanantha321 in https://github.com/kserve/kserve/pull/4099
• Replace klog with klog/v2 by @sivanantha321 in https://github.com/kserve/kserve/pull/4093
• Add exception handling and logging for grpc server by @sivanantha321 in https://github.com/kserve/kserve/pull/4066
• Update ClusterLocalModel to LocalModelCache by @yuzisun in https://github.com/kserve/kserve/pull/4105
• Fix LocalModelCache controller reconciles deleted resource by @sivanantha321 in https://github.com/kserve/kserve/pull/4106
• Fix InferenceService state when Predictor pod in CrashLoopBackOff by @hdefazio in https://github.com/kserve/kserve/pull/4003
• LocalModelCache Admission Webhook by @HotsauceLee in https://github.com/kserve/kserve/pull/4102
• Add namespace to localmodel and localmodelnode ServiceAccount helm chart by @ritzdevp in https://github.com/kserve/kserve/pull/4111
• KServe VLLM cpu image by @AyushSawant18588 in https://github.com/kserve/kserve/pull/4049
• Update max_model_len calculation and fixup encoder pooling by @Datta0 in https://github.com/kserve/kserve/pull/4055
• chore: use patch instead of update for finalizer changes by @whynowy in https://github.com/kserve/kserve/pull/4072
• Fix isvc role localmodelcache permission by @sivanantha321 in https://github.com/kserve/kserve/pull/4131
• Detect missing models and redownload models by @greenmoon55 in https://github.com/kserve/kserve/pull/4095
• introduce service configuration at configmap level by @spolti in https://github.com/kserve/kserve/pull/3672
• Allow multiple node groups in the model cache CR by @greenmoon55 in https://github.com/kserve/kserve/pull/4134
• Annotation to disable model cache by @greenmoon55 in https://github.com/kserve/kserve/pull/4118
• Clean up jobs in model cache agent by @greenmoon55 in https://github.com/kserve/kserve/pull/4140
• Ensure Model root folder exists by @greenmoon55 in https://github.com/kserve/kserve/pull/4142
• Add NodeGroup Name Into PVC Name by @HotsauceLee in https://github.com/kserve/kserve/pull/4141
• Make LocalModel Agent reconcilation frequency configurable by @greenmoon55 in https://github.com/kserve/kserve/pull/4143
• Remove deepcopy-gen in favour of controller-gen by @sivanantha321 in https://github.com/kserve/kserve/pull/4109
• Add ability to set annotations on controll/webhook service and expose metrics bind port and address in helm chart by @mhowell24 in https://github.com/kserve/kserve/pull/4127
• Fix EOF error for downloading zip files by @Jonas-Bruns in https://github.com/kserve/kserve/pull/4082
• Remove redundant namespace yaml by @greenmoon55 in https://github.com/kserve/kserve/pull/4148
• Fix Localmodel agent build by @greenmoon55 in https://github.com/kserve/kserve/pull/4150
• Fix model server fails to gracefully shutdown by @sivanantha321 in https://github.com/kserve/kserve/pull/4116
• Ensure root model directory exists and add protection for jobs created by @yuzisun in https://github.com/kserve/kserve/pull/4152
• Enable transformer deeper readiness check tests by @sivanantha321 in https://github.com/kserve/kserve/pull/4121
• Update HuggingFace server dependencies versions by @AyushSawant18588 in https://github.com/kserve/kserve/pull/4147
• Add workflow for verifying go mod by @sivanantha321 in https://github.com/kserve/kserve/pull/4137
• Fix for CVE-2024-52304 - aiohttp upgrade by @andyi2it in https://github.com/kserve/kserve/pull/4113
• Allow other engine builders other than docker by @spolti in https://github.com/kserve/kserve/pull/3906
• Add localmodelnode crd to helm chart by @greenmoon55 in https://github.com/kserve/kserve/pull/4161
• Fixes Non-linear parsing of case-insensitive content by @spolti in https://github.com/kserve/kserve/pull/4158
• Helm chart - option to run daemonset as root by @greenmoon55 in https://github.com/kserve/kserve/pull/4164
• Replace nodeGroup with nodeGroups in charts/kserve-crd by @ritzdevp in https://github.com/kserve/kserve/pull/4166
• Add affinity and tolerations to localmodel daemonset by @greenmoon55 in https://github.com/kserve/kserve/pull/4173
• Fix s3 download PermanentRedirectError for legacy s3 endpoint by @bentohset in https://github.com/kserve/kserve/pull/4157
• Make label and annotation propagation configurable by @spolti in https://github.com/kserve/kserve/pull/4030
• Add ModelCache e2e test by @sivanantha321 in https://github.com/kserve/kserve/pull/4136
• Update vllm to 0.6.6 by @rajatvig in https://github.com/kserve/kserve/pull/4176
• [bugfix] fix s3 storage download filename bug by @anencore94 in https://github.com/kserve/kserve/pull/4162
• Add hf to storageuri prefix list by @tjandy98 in https://github.com/kserve/kserve/pull/4184
• Add Support for OpenAI-compatible Embeddings API by @FabianScheidt in https://github.com/kserve/kserve/pull/4129
• fix: typo in _construct_http_status_error method by @Mgla96 in https://github.com/kserve/kserve/pull/4190
• Fix raw logger e2e test by @sivanantha321 in https://github.com/kserve/kserve/pull/4185
• Feat: Support configuring isvc resource defaults by @andyi2it in https://github.com/kserve/kserve/pull/4032
• keep replicas when autoscaler set external by @Jooho in https://github.com/kserve/kserve/pull/4196
• Increase kserve controller readiness probe time period by @sivanantha321 in https://github.com/kserve/kserve/pull/4200
• Fix golangci-lint binary path selection based on GOBIN by @Jooho in https://github.com/kserve/kserve/pull/4198
• Add option to disable volume management in localModel config by @ritzdevp in https://github.com/kserve/kserve/pull/4186
• set MaxUnavailable(0%)/MaxSurge(100%) for rollingUpdate in multinode case by @Jooho in https://github.com/kserve/kserve/pull/4188
• Gracefully shutdown the router server by @sivanantha321 in https://github.com/kserve/kserve/pull/3367
• Add workflow for manual huggingface vLLM image publish by @sivanantha321 in https://github.com/kserve/kserve/pull/4092
• Feat: Gateway API Support - Raw Deployment by @sivanantha321 in https://github.com/kserve/kserve/pull/3952
• add make goal to build huggingface cpu image by @spolti in https://github.com/kserve/kserve/pull/4202
• Cleanup the filepath in createNewFile to avoid path traversal issue by @hdefazio in https://github.com/kserve/kserve/pull/4205
• Enhance multinode health_check python and manifests by @Jooho in https://github.com/kserve/kserve/pull/4197
• Publish 0.15-rc0 release by @yuzisun in https://github.com/kserve/kserve/pull/4213
• Fix Gateway API flaky test by @sivanantha321 in https://github.com/kserve/kserve/pull/4214
• Remove linux/arm64/v8 as platform option to fix build errors by @gavrissh in https://github.com/kserve/kserve/pull/4217
• Fix: typo in inferenceservice configmap by @sukumargaonkar in https://github.com/kserve/kserve/pull/4215
• Fix CI not using localmodelnode agent dev image by @sivanantha321 in https://github.com/kserve/kserve/pull/4221
• Fix model download path by @hakuro95 in https://github.com/kserve/kserve/pull/4112
• Support Multiple NodeGroups In LocalModelCache by @HotsauceLee in https://github.com/kserve/kserve/pull/4170
• Inference Graph: use plain text HTTP when part of Istio Mesh by @israel-hdez in https://github.com/kserve/kserve/pull/4031
• Better compatibility with in-place upgrades by @israel-hdez in https://github.com/kserve/kserve/pull/4234
• Increase request timeout seconds for art explainer by @sivanantha321 in https://github.com/kserve/kserve/pull/4241
• fix: add trainedmodels custom resource to kubeflow-kserve clusterroles by @gigabyte132 in https://github.com/kserve/kserve/pull/4225
• Fix CVE-2025-24357 and Bump vLLM to 0.7.2 by @sivanantha321 in https://github.com/kserve/kserve/pull/4223
• Use Go 1.23 to build kserve and update mod versions by @rajatvig in https://github.com/kserve/kserve/pull/4239
• install: Remove modelmesh installation from helm chart by @sivanantha321 in https://github.com/kserve/kserve/pull/4243
• Bump golang-lint to 1.63 and fix all linter errors by @sivanantha321 in https://github.com/kserve/kserve/pull/3967
• Issue 4248: Request Logger with Multiple Metadata Headers fail by @tylerhyang in https://github.com/kserve/kserve/pull/4249
• Add predictor healthcheck to OpenAIProxyModel by @greenmoon55 in https://github.com/kserve/kserve/pull/4250
• Expose podSpec fields for Inferencegraph by @sivanantha321 in https://github.com/kserve/kserve/pull/4091
• Fix localmodel test by @greenmoon55 in https://github.com/kserve/kserve/pull/4268
• Force symlink for ModelCar by @pmtk in https://github.com/kserve/kserve/pull/4274
• Refactor vLLM + Embed support by @gavrissh in https://github.com/kserve/kserve/pull/4177
• Fix triton health check by @greenmoon55 in https://github.com/kserve/kserve/pull/4277
• Upgrade vLLM version to 0.7.3 by @gavrissh in https://github.com/kserve/kserve/pull/4281
• 0.15.0-rc1 release by @greenmoon55 in https://github.com/kserve/kserve/pull/4285
• Add model_version field to InferRequest by @greenmoon55 in https://github.com/kserve/kserve/pull/4287
• (Bug [#4273]) quick_install.sh failed to uninstall incomplete installation and has small syntax bug by @zozowell in https://github.com/kserve/kserve/pull/4275
• update openshift guide by @spolti in https://github.com/kserve/kserve/pull/4210
• Collocation transformer and predictor spec by @sivanantha321 in https://github.com/kserve/kserve/pull/4255
• Move arguments from 'args' to 'command' for huggingface server multnode SR by @Jooho in https://github.com/kserve/kserve/pull/4289
• Include reasoning parser option in vLLM for reasoning models by @gavrissh in https://github.com/kserve/kserve/pull/4282
• KServe Keda Integration by @andyi2it in https://github.com/kserve/kserve/pull/3652
• add huggingfaceserver-multinode to helm chart by @Jooho in https://github.com/kserve/kserve/pull/4293
• Add missing CRDs for Keda by @andyi2it in https://github.com/kserve/kserve/pull/4296
• Add well-known label to InferenceGraphs by @israel-hdez in https://github.com/kserve/kserve/pull/4236
• Docs update for speculative_decoding in huggingfaceserver by @nickaggarwal in https://github.com/kserve/kserve/pull/4272
• Merging kfp into kserve docs. by @mholder6 in https://github.com/kserve/kserve/pull/4270
• fix conflict isvc name for keda e2e by @Jooho in https://github.com/kserve/kserve/pull/4305
• update golangci-lint verstion to 1.64 by @Jooho in https://github.com/kserve/kserve/pull/4302
• [bug] add huggingfaceserver gpu images to kustomization.yaml and use it in multinode sr by @Jooho in https://github.com/kserve/kserve/pull/4306
• Fix watch for k8s service events by @sivanantha321 in https://github.com/kserve/kserve/pull/4260
• Fix rest server multiprocess workers not working by @sivanantha321 in https://github.com/kserve/kserve/pull/4271
• Stop token generation when client is disconnected by @sivanantha321 in https://github.com/kserve/kserve/pull/4307
• Combine precommit checks by @sivanantha321 in https://github.com/kserve/kserve/pull/4256
• [Bug] Default "dtype" was overriding vLLM "auto" dtype mode by @gavrissh in https://github.com/kserve/kserve/pull/4313
• Make webhook metrics aggregate injector idempotent by @javierdlrm in https://github.com/kserve/kserve/pull/4333
• Fix: Model cache uri requires subpath match instead of string match by @greenmoon55 in https://github.com/kserve/kserve/pull/4326
• Refactor: Consolidate duplicate custom gpu type checking logic by @Jooho in https://github.com/kserve/kserve/pull/4303
• fix: Static LoRA modules are not initialized by the Huggingface Server by @sivanantha321 in https://github.com/kserve/kserve/pull/4339
• CPU Dockerfile with Intel Extension for PyTorch and oneDNN support by @ashahba in https://github.com/kserve/kserve/pull/4280
• Keda integration external scaler by @andyi2it in https://github.com/kserve/kserve/pull/4295
• Upgrade vLLM to v0.8.1 by @gavrissh in https://github.com/kserve/kserve/pull/4340
• fix: Hostname Generation for Additional Ingress Domains in RawDeployment Mode by @sivanantha321 in https://github.com/kserve/kserve/pull/4336
• KServe 0.15.0 by @greenmoon55 in https://github.com/kserve/kserve/pull/4347
• chore: Remove redundant python lint/style workflow by @sivanantha321 in https://github.com/kserve/kserve/pull/4330
• clean up autoscaling code for raw deployment by @yuzisun in https://github.com/kserve/kserve/pull/4349

New Contributors

• @ArjunBhalla98 made their first contribution in https://github.com/kserve/kserve/pull/4033
• @LOADBC made their first contribution in https://github.com/kserve/kserve/pull/4012
• @ops-jaeha made their first contribution in https://github.com/kserve/kserve/pull/4018
• @kursataktas made their first contribution in https://github.com/kserve/kserve/pull/4038
• @oplushappy made their first contribution in https://github.com/kserve/kserve/pull/4024
• @carlory made their first contribution in https://github.com/kserve/kserve/pull/4051
• @bentohset made their first contribution in https://github.com/kserve/kserve/pull/4064
• @ritzdevp made their first contribution in https://github.com/kserve/kserve/pull/4111
• @AyushSawant18588 made their first contribution in https://github.com/kserve/kserve/pull/4049
• @whynowy made their first contribution in https://github.com/kserve/kserve/pull/4072
• @mhowell24 made their first contribution in https://github.com/kserve/kserve/pull/4127
• @Jonas-Bruns made their first contribution in https://github.com/kserve/kserve/pull/4082
• @rajatvig made their first contribution in https://github.com/kserve/kserve/pull/4176
• @FabianScheidt made their first contribution in https://github.com/kserve/kserve/pull/4129
• @Mgla96 made their first contribution in https://github.com/kserve/kserve/pull/4190
• @hakuro95 made their first contribution in https://github.com/kserve/kserve/pull/4112
• @gigabyte132 made their first contribution in https://github.com/kserve/kserve/pull/4225
• @tylerhyang made their first contribution in https://github.com/kserve/kserve/pull/4249
• @pmtk made their first contribution in https://github.com/kserve/kserve/pull/4274
• @zozowell made their first contribution in https://github.com/kserve/kserve/pull/4275
• @nickaggarwal made their first contribution in https://github.com/kserve/kserve/pull/4272
• @javierdlrm made their first contribution in https://github.com/kserve/kserve/pull/4333
• @ashahba made their first contribution in https://github.com/kserve/kserve/pull/4280

Full Changelog: https://github.com/kserve/kserve/compare/v0.14.1...v0.15.0