Gaia Scheduler \u53ef\u4ee5\u5206\u4e3a 4 \u4e2a\u7ec4\u4ef6\uff1a<\/p>\n
- \n
- GPU Manager: \u4f5c\u4e3a device plugin \u5411 kubelet \u6ce8\u518c\u3002\u5171\u6ce8\u518c\u4e86\u4e24\u4e2a\u8bbe\u5907\uff0c\u5305\u62ec vcore \u548c vmemory\uff0c\u652f\u6301\u4e24\u79cd\u8ba1\u7b97\u8d44\u6e90\uff1a
tencent.com\/vcuda-core<\/code> \u548ctencent.com\/vcuda-memory<\/code>\uff0c\u5206\u522b\u7528\u6765\u505a GPU \u8ba1\u7b97\u8d44\u6e90\u548c GPU \u5185\u5b58\u8d44\u6e90\u7684\u8bf7\u6c42\u548c\u9650\u5236\u3002<\/p>\n<\/li>\n- \n
GPU Scheduler: \u8fd9\u91cc\u7684 scheduler \u5e76\u4e0d\u662f kubernetes \u7684\u8c03\u5ea6\u5668\uff0c\u662f GPU Manager \u5728\u6536\u5230 kubelet \u7684 Allocate \u8c03\u7528\u540e\uff0c\u5b83\u9700\u6c42\u5c06\u8bbe\u5907\u6302\u8f7d\u7ed9\u5bb9\u5668\u3002\u4e3a\u4e86\u5b9e\u73b0\u6700\u4f73\u7684 GPU \u6302\u8f7d\uff0c\u5c31\u6709\u8fd9\u6837\u4e00\u4e2a\u4e13\u95e8\u7684 Scheduler \u6765\u6839\u636e\u8282\u70b9\u4e0a\u5f53\u524d\u7684 GPU \u62d3\u6251\u548c\u8d44\u6e90\u5360\u7528\u60c5\u51b5\u8fdb\u884c\u8c03\u5ea6\u3002<\/p>\n<\/li>\n
- \n
vGPU Manager: vGPU Manager \u662f\u5177\u4f53\u8d1f\u8d23\u7ba1\u7406\u5bb9\u5668\u7684\u7ec4\u4ef6\uff0c\u5305\u62ec\u76d1\u63a7\u5bb9\u5668\u72b6\u6001\uff0c\u4f20\u9012\u914d\u7f6e\uff0c\u548c\u5bb9\u5668\u5185\u7684vGPU Library\u901a\u4fe1\uff0c\u4ee5\u53ca\u5728\u5bb9\u5668\u6b7b\u4ea1\u540e\u8fdb\u884c\u56de\u6536\u64cd\u4f5c\u3002<\/p>\n<\/li>\n
- \n
vGPU Library: vGPU Library \u867d\u7136\u76f8\u5173\u7684\u4ee3\u7801\u91cf\u4e0d\u591a\uff0c\u4f46\u5b83\u662f Gaia Scheduler \u6700\u91cd\u8981\u7684\u90e8\u5206\u3002\u56e0\u4e3a\u5b83\u662f\u5b9e\u73b0 GPU \u865a\u62df\u5316\u7684\u6838\u5fc3\u3002\u901a\u8fc7\u8986\u76d6\u5bb9\u5668\u4e2d\u7684 LD_LIBRARY_PATH \u4ee5\u53ca\u81ea\u5b9a\u4e49\u4e86
libcuda-control.so<\/code> \u5b9e\u73b0\u5bf9 CUDA API \u7684\u62e6\u622a\u3002<\/p>\n<\/li>\n<\/ul>\nGaia Scheduler \u4e3b\u8981\u7531\u4e09\u4e2a\u9879\u76ee\u7ec4\u6210: gpu-manager<\/a> \u548c vcuda-controller<\/a>\uff0cgpu-admission<\/a>\u3002\u4f46\u662f\u8fd9\u91cc\u7684 gpu-manager \u662f Gaia Scheduler \u7684\u4e3b\u8981\u5b9e\u73b0\uff0c\u5305\u542b\u4e86\u4e0a\u8ff0\u7684 4 \u4e2a\u7ec4\u4ef6\uff0cvcuda-controller \u5c31\u662f vGPU Library\uff0c\u5df2\u7ecf\u88ab\u6253\u5305\u5230\u4e86 gpu-manager \u8fd9\u4e2a\u9879\u76ee\u4e2d\u3002gpu-manager \u9700\u8981\u914d\u5408 gpu-admission \u9879\u76ee\u6765\u5b8c\u6210 GPU Scheduler \u7684\u5de5\u4f5c\u3002\u4e0d\u8981\u56e0\u6b64\u4ea7\u751f\u8bef\u89e3\u3002\u4e0b\u6587\u4e2d\u6211\u4eec\u4e3b\u8981\u5c31 gpu-manager \u8fd9\u4e2a\u9879\u76ee\u8fdb\u884c\u5206\u6790\u3002<\/p>\n
\u542f\u52a8\u6d41\u7a0b\u5206\u6790<\/h2>\n
gpu-manager \u672c\u8eab\u4e3b\u8981\u4f5c\u4e3a kubernetes \u7684 device plugin \u6765\u5b9e\u73b0\u7684\uff0c\u5b9a\u4e49\u4e86\u4e24\u79cd\u8bbe\u5907:
vcuda-core<\/code> \u548cvcuda-memory<\/code>\uff0c\u6211\u4eec\u7684\u5e94\u7528\u901a\u8fc7 pod \u7684\u8d44\u6e90\u5b57\u6bb5\u8fdb\u884c\u7533\u8bf7\uff0c\u7136\u540e kube-scheduler \u4f1a\u6839\u636e\u8282\u70b9\u4e0a\u7684\u8d44\u6e90\u72b6\u6001\u8fdb\u884c\u8c03\u5ea6\u3002\u56e0\u6b64\uff0c\u4f60\u6700\u597d\u8fd8\u9700\u8981\u4e86\u89e3 kubernetes \u7684 device plugin \u7684\u5f00\u53d1\u77e5\u8bc6\u3002\u5173\u4e8e device plugin \u7684\u5f00\u53d1\uff0c\u53ef\u4ee5\u770b\u4e4b\u524d\u7684\u4e00\u7bc7\u6587\u7ae0\uff1aKubernetes\u5f00\u53d1\u77e5\u8bc6–device-plugin\u7684\u5b9e\u73b0<\/a>\u3002<\/p>\n\u542f\u52a8\u53c2\u6570<\/h3>\n
\u5206\u6790\u4e00\u4e2a\u9879\u76ee\u4ece\u542f\u52a8\u53c2\u6570\u5f00\u59cb\uff0c\u53ef\u4ee5\u5e2e\u52a9\u6211\u4eec\u5feb\u901f\u4e86\u89e3\uff1a<\/p>\n
- \n
- driver: \u8fd9\u4e2a\u662f GPU \u7684\u9a71\u52a8\uff0c\u5f53\u524d\u7684\u9ed8\u8ba4\u503c\u662f nvidia\uff0c\u5f88\u663e\u7136\u8be5\u9879\u76ee\u53ef\u4ee5\u6269\u5c55\u652f\u6301\u5176\u4ed6\u7c7b\u578b\u7684 GPU\u3002<\/li>\n
- extra-config: \u989d\u5916\u7684\u914d\u7f6e\uff0c\u8fd9\u4e2a\u53c2\u6570\u6682\u65f6\u770b\u4e0d\u51fa\u6765\u6709\u4ec0\u4e48\u7279\u522b<\/li>\n
- volume-config: \u8fd9\u91cc\u7684 volume \u6307\u7684\u662f\u4e00\u4e9b\u52a8\u6001\u94fe\u63a5\u5e93\u548c\u53ef\u6267\u884c\u6587\u4ef6\u7684\u4f4d\u7f6e\u3002\u4e5f\u5c31\u662f gpu-manager \u9700\u8981\u62e6\u622a\u8c03\u7528\u7684\u4e00\u4e9b\u5e93<\/li>\n
- docker-endpoint: \u7528\u6765\u6302\u8f7d\u5230\u5bb9\u5668\u4e2d\u548c docker \u505a\u901a\u4fe1\u7684\uff0c\u9ed8\u8ba4\u4f4d\u7f6e\u662f
unix:\/\/\/\/var\/run\/docker.sock<\/code><\/li>\n- query-port: \u7edf\u8ba1\u4fe1\u606f\u670d\u52a1\u7684\u67e5\u8be2\u63a5\u53e3<\/li>\n
- query-port: \u7edf\u8ba1\u4fe1\u606f\u670d\u52a1\u7684\u76d1\u542c\u5730\u5740<\/li>\n
- kubeconfig: \u7528\u6765\u6388\u6743\u7684\u914d\u7f6e\u6587\u4ef6<\/li>\n
- standalone: \u6682\u65f6\u8fd8\u4e0d\u6e05\u695a\u7684\u53c2\u6570<\/li>\n
- sample-period: gpu-manager \u4f1a\u67e5\u8be2 gpu \u8bbe\u5907\u7684\u4f7f\u7528\u60c5\u51b5\uff0c\u8fd9\u4e2a\u53c2\u6570\u7528\u6765\u8bbe\u5b9a\u91c7\u6837\u5468\u671f<\/li>\n
- node-labels: \u7ed9\u8282\u70b9\u81ea\u52a8\u6253\u6807\u7b7e<\/li>\n
- hostname-override: gpu-manager \u5728\u8fd0\u884c\u65f6\uff0c\u53ea\u5173\u6ce8\u81ea\u5df1\u8282\u70b9\u4e0a\u7684 pod\uff0c\u8fd9\u4e3b\u8981\u662f\u901a\u8fc7 hostname \u6765\u8fa8\u8ba4\u7684<\/li>\n
- virtual-manager-path: gpu-manager \u4f1a\u4e3a\u6240\u6709\u9700\u8981\u865a\u62df gpu \u8d44\u6e90\u7684 pod \u521b\u5efa\u552f\u4e00\u7684\u6587\u4ef6\u5939\uff0c\u6587\u4ef6\u5939\u7684\u8def\u5f84\u5c31\u5728\u8fd9\u4e2a\u5730\u5740\u4e0b\u3002<\/li>\n
- device-plugin-path: kubernetes \u9ed8\u8ba4\u7684 device plugin \u7684\u76ee\u5f55\u5730\u5740<\/li>\n
- checkpoint-path: gpu-manager \u4f1a\u4ea7\u751f checkpoint \u6765\u5f53\u7f13\u5b58\u7528<\/li>\n
- share-mode: gpu-manager \u6700\u5927\u7684\u7279\u70b9\u5c31\u662f\u5c06\u4e00\u4e2a\u7269\u7406 gpu \u5206\u6210\u591a\u4e2a\u865a\u62df gpu\uff0c\u4e5f\u5c31\u662f\u5171\u4eab\u6a21\u5f0f<\/li>\n
- allocation-check-period: \u68c0\u67e5\u5206\u914d\u4e86\u865a\u62df gpu \u8d44\u6e90\u7684 pod \u7684\u72b6\u6001\uff0c\u53ca\u65f6\u56de\u6536\u8d44\u6e90<\/li>\n
- incluster-mode: \u662f\u5426\u5728\u96c6\u7fa4\u5185\u8fd0\u884c<\/li>\n<\/ul>\n
\u670d\u52a1\u542f\u52a8<\/h3>\n
gpu-manager \u63a8\u8350\u7684\u90e8\u7f72\u65b9\u6848\u662f\u901a\u8fc7 kubernetes \u7684 daemonset\uff0c\u7136\u540e\u914d\u7f6e node selector \u8c03\u5ea6\u5230\u6307\u5b9a\u7684\u8282\u70b9\u4e0a\u3002\u7136\u540e gpu-manager \u5c31\u5f00\u59cb\u5728\u6307\u5b9a\u8282\u70b9\u4e0a\u542f\u52a8\u4e86\u3002<\/p>\n
srv := server.NewManager(cfg)\ngo srv.Run()\n<\/code><\/pre>\n\u8fd9\u91cc\uff0c\u6211\u4eec\u9700\u8981\u770b\u4e00\u4e0b\u8fd9\u4e2a srv \u7684\u5177\u4f53\u5b9e\u73b0\uff0c\u9996\u5148\u662f\u5b83\u7684\u7ed3\u6784\u4f53\uff1a<\/p>\n
type managerImpl struct {\n config *config.Config\n\n allocator allocFactory.GPUTopoService \/\/ gpu \u5bb9\u5668\u8c03\u5ea6\u5206\u914d\n displayer *display.Display \/\/ gpu \u4f7f\u7528\u60c5\u51b5\u53ef\u89c6\u5316\u670d\u52a1\n virtualManager *vitrual_manager.VirtualManager \/\/ \u8d1f\u8d23\u7ba1\u7406 vgpu\n\n bundleServer map[string]ResourceServer\n srv *grpc.Server\n}\n<\/code><\/pre>\nconfig \u5305\u542b\u4e86\u6211\u4eec\u4e0a\u9762\u7684\u6240\u6709\u53c2\u6570\uff0c\u5c31\u4e0d\u8fdb\u53bb\u7ec6\u770b\u4e86\u3002<\/p>\n
allocator \u8d1f\u8d23\u5728\u5bb9\u5668\u8c03\u5ea6\u5230\u8282\u70b9\u4e0a\u540e\uff0c\u4e3a\u5176\u5206\u914d\u5177\u4f53\u7684\u8bbe\u5907\u8d44\u6e90\u3002allocator \u5b9e\u73b0\u4e86\u63a2\u6d4b\u8282\u70b9\u4e0a\u7684 gpu \u62d3\u6251\u67b6\u6784\uff0c\u7136\u540e\u4ee5\u6700\u4f73\u6027\u80fd\uff0c\u6700\u5c11\u788e\u7247\u4e3a\u76ee\u7684\u4f7f\u7528\u6700\u4f18\u7684\u65b9\u6848\u8fdb\u884c\u8d44\u6e90\u5206\u914d\u3002<\/p>\n
displayer \u662f\u5c06 gpu \u7684\u4f7f\u7528\u60c5\u51b5\u8f93\u51fa\uff0c\u65b9\u4fbf\u6211\u4eec\u67e5\u770b\u3002<\/p>\n
virtualManager \u8d1f\u8d23 vgpu \u5206\u914d\u540e\u7684\u7ba1\u7406\u5de5\u4f5c\u3002<\/p>\n
bundleServer \u5305\u542b vcore\uff0cvmemory\uff0c\u6211\u4eec\u4e0a\u9762\u63d0\u5230\u8fd9\u4e24\u79cd\u8d44\u6e90\u4ee5 device plugin \u7684\u65b9\u5f0f\u8fdb\u884c\u6ce8\u518c\uff0c\u56e0\u6b64\u4ed6\u4eec\u9700\u8981\u542f\u52a8 grpc server\u3002<\/p>\n
srv: \u5c06 gpu display server \u6ce8\u518c\u5230\u8fd9\u4e2a grpc server \u4e2d\u3002<\/p>\n
\u63a5\u4e0b\u6765\uff0c\u6211\u4eec\u5c31\u53ef\u4ee5\u5206\u6790
srv.Run()<\/code> \u65b9\u6cd5\u5177\u4f53\u6267\u884c\u4e86\u54ea\u4e9b\u5185\u5bb9\u3002\u4e3a\u4e86\u5148\u5bf9\u6574\u4e2a\u6d41\u7a0b\u6709\u4e2a\u5927\u6982\u7684\u5370\u8c61\uff0c\u6211\u5c06\u5185\u5bb9\u6574\u7406\u6210\u4ee5\u4e0b\u6761\u76ee\uff1a<\/p>\n- \n
- \u542f\u52a8 volumeManager\uff0c\u5c06\u8282\u70b9\u4e0a\u548c nvidia gpu (\u5305\u62eccuda) \u7684\u6240\u6709\u53ef\u6267\u884c\u6587\u4ef6\u548c\u5e93\u79fb\u52a8\u5230 \/etc\/gpu-manager\/vdriver \u4e2d\u3002\u5e76\u4e14\u5c06\u5173\u952e\u7684\u5e93\u66ff\u6362\u6210 vcuda-control\uff0c\u5b9e\u73b0 cuda \u8c03\u7528\u7684\u62e6\u622a\u3002<\/li>\n
- watchdog \u521b\u5efa pod \u7f13\u5b58\u5e76\u76d1\u63a7 pod\uff0c\u4e4b\u540e\u6240\u6709\u5173\u4e8e pod \u7684\u64cd\u4f5c\u90fd\u6765\u6e90\u4e8e\u8fd9\u91cc\u3002<\/li>\n
- watchdog \u7ed9\u8282\u70b9\u6253\u4e0a\u6807\u7b7e<\/li>\n
- \u542f\u52a8 virtualManager<\/li>\n
- gpu \u62d3\u6251\u7ed3\u6784\u611f\u77e5\u3002<\/li>\n
- \u521d\u59cb\u5316\u8d44\u6e90\u5206\u914d\u5668<\/li>\n
- \u8bbe\u7f6e vcuda, vmemory, display \u7684 grpc \u670d\u52a1<\/li>\n
- \u542f\u52a8 metrics \u7684 http \u670d\u52a1\uff0c\u4e3b\u8981\u662f\u63d0\u4f9b\u7ed9 prometheus<\/li>\n
- \u542f\u52a8 vcuda\uff0cvmemory \u7684 grpc \u670d\u52a1<\/li>\n
- \u542f\u52a8 display \u7684 grpc \u670d\u52a1<\/li>\n<\/ul>\n
\u63a5\u4e0b\u6765\uff0c\u6211\u4eec\u5177\u4f53\u6765\u5206\u6790\u6bcf\u4e00\u6b65\u662f\u5982\u4f55\u505a\u7684\u3002\u5f53\u7136\uff0c\u8fd9\u91cc\u53ea\u4f1a\u6311\u4e00\u4e9b\u91cd\u70b9\u7684\u90e8\u5206\u3002<\/p>\n
volumeManager \u7684\u542f\u52a8<\/h4>\n
func (vm *VolumeManager) Run() (err error) {\n \/\/ ldcache \u662f\u52a8\u6001\u94fe\u63a5\u5e93\u7684\u7f13\u5b58\u4fe1\u606f\n cache, err := ldcache.Open()\n defer func() {\n if e := cache.Close(); err == nil {\n err = e\n }\n }()\n vols := make(VolumeMap)\n for _, cfg := range vm.Config {\n vol := &Volume{\n Path: path.Join(cfg.BasePath, cfg.Name),\n }\n\n if cfg.Name == \"nvidia\" {\n \/\/ nvidia \u5e93\u7684\u4f4d\u7f6e\n types.DriverLibraryPath = filepath.Join(cfg.BasePath, cfg.Name)\n } else {\n \/\/ origin \u5e93\u7684\u4f4d\u7f6e\n types.DriverOriginLibraryPath = filepath.Join(cfg.BasePath, cfg.Name)\n }\n\n for t, c := range cfg.Components {\n switch t {\n case \"binaries\":\n \/\/ \u8c03\u7528 which \u6765\u67e5\u627e\u53ef\u6267\u884c\u6587\u4ef6\u7684\u4f4d\u7f6e\n bins, err := which(c...)\n \/\/ \u5c06\u5b9e\u9645\u4f4d\u7f6e\u5b58\u8d77\u6765\n vol.dirs = append(vol.dirs, volumeDir{binDir, bins})\n case \"libraries\":\n \/\/ \u662f\u5e93\u7684\u8bdd\uff0c\u5c31\u4ece ldcache \u91cc\u9762\u53bb\u627e\n libs32, libs64 := cache.Lookup(c...)\n \/\/ \u5c06 library \u4f4d\u7f6e\u5b58\u8d77\u6765\n vol.dirs = append(vol.dirs, volumeDir{lib32Dir, libs32}, volumeDir{lib64Dir, libs64})\n }\n vols[cfg.Name] = vol\n }\n }\n \/\/ \u627e\u5230\u4e86\u9700\u8981\u7684\u5e93\u4f4d\u7f6e\u4e4b\u540e\uff0c\u505a mirror \u5904\u7406\n if err := vm.mirror(vols); err != nil {\n return err\n }\n return nil\n}\n<\/code><\/pre>\n\u8fd9\u6bb5\u4ee3\u7801\u7684\u524d\u534a\u90e8\u5206\u90fd\u662f\u5728\u67e5\u627e\u6307\u5b9a\u7684\u52a8\u6001\u94fe\u63a5\u5e93\u548c\u53ef\u6267\u884c\u6587\u4ef6\uff0c\u8fd9\u4e9b\u6587\u4ef6\u662f\u5728 volume.conf \u8fd9\u4e2a\u914d\u7f6e\u6587\u4ef6\u4e2d\u6307\u5b9a\u7684\uff0c\u901a\u8fc7\u53c2\u6570\u4f20\u8fdb\u6765\u3002\u67e5\u627e\u52a8\u6001\u94fe\u63a5\u5e93\u65f6\uff0c\u4f7f\u7528\u7684\u662f ldcache\uff0c\u67e5\u627e\u53ef\u6267\u884c\u6587\u4ef6\u65f6\uff0c\u4f7f\u7528\u4e86\u7cfb\u7edf\u7684
which<\/code> \u6307\u4ee4\u3002\u627e\u5230\u4e4b\u540e\u4f1a\u5c06\u5176\u6240\u5728\u4f4d\u7f6e\u8bb0\u5f55\u4e0b\u6765\u3002\u63a5\u7740\u5c31\u662f\u5bf9\u627e\u5230\u7684\u5e93\u505amirror<\/code> \u5904\u7406\u3002<\/p>\nfunc (vm *VolumeManager) mirror(vols VolumeMap) error {\n \/\/ nvidia \u548c origin\n for driver, vol := range vols {\n if exist, _ := vol.exist(); !exist {\n \/\/ \u8fd9\u91cc\u7684path\u662f\/etc\/gpu-manager\/vdriver\u4e0b\u9762\n if err := os.MkdirAll(vol.Path, 0755); err != nil {\n return err\n }\n }\n for _, d := range vol.dirs {\n vpath := path.Join(vol.Path, d.name)\n \/\/ \u521b\u5efa bin lib lib64\n if err := os.MkdirAll(vpath, 0755); err != nil {\n return err\n }\n\n \/\/ For each file matching the volume components (blacklist excluded), create a hardlink\/copy\n \/\/ of it inside the volume directory. We also need to create soname symlinks similar to what\n \/\/ ldconfig does since our volume will only show up at runtime.\n for _, f := range d.files {\n glog.V(2).Infof(\"Mirror %s to %s\", f, vpath)\n if err := vm.mirrorFiles(driver, vpath, f); err != nil {\n return err\n }\n\n if strings.HasPrefix(path.Base(f), \"libcuda.so\") {\n driverStr := strings.SplitN(strings.TrimPrefix(path.Base(f), \"libcuda.so.\"), \".\", 2)\n types.DriverVersionMajor, _ = strconv.Atoi(driverStr[0]) \/\/ \u9a71\u52a8\u7248\u672c\u53f7\n types.DriverVersionMinor, _ = strconv.Atoi(driverStr[1])\n glog.V(2).Infof(\"Driver version: %d.%d\", types.DriverVersionMajor, types.DriverVersionMinor)\n }\n\n if strings.HasPrefix(path.Base(f), \"libcuda-control.so\") {\n vm.cudaControlFile = f\n }\n }\n }\n }\n\n vCudaFileFn := func(soFile string) error {\n if err := os.Remove(soFile); err != nil {\n if !os.IsNotExist(err) {\n return err\n }\n }\n if err := clone(vm.cudaControlFile, soFile); err != nil {\n return err\n }\n\n glog.V(2).Infof(\"Vcuda %s to %s\", vm.cudaControlFile, soFile)\n\n l := strings.TrimRight(soFile, \".0123456789\")\n if err := os.Remove(l); err != nil {\n if !os.IsNotExist(err) {\n return err\n }\n }\n if err := clone(vm.cudaControlFile, l); err != nil {\n return err\n }\n glog.V(2).Infof(\"Vcuda %s to %s\", vm.cudaControlFile, l)\n return nil\n }\n\n if vm.share && len(vm.cudaControlFile) > 0 {\n if len(vm.cudaSoname) > 0 {\n for _, f := range vm.cudaSoname {\n if err := vCudaFileFn(f); err != nil {\n return err\n }\n }\n }\n\n if len(vm.mlSoName) > 0 {\n for _, f := range vm.mlSoName {\n if err := vCudaFileFn(f); err != nil {\n return err\n }\n }\n }\n }\n\n return nil\n}\n<\/code><\/pre>\n\u8fd9\u6bb5\u4ee3\u7801\u5148\u4f1a\u5bf9\u6240\u6709\u4e0a\u9762\u67e5\u627e\u5230\u7684\u5e93\u6216\u53ef\u6267\u884c\u6587\u4ef6\u8c03\u7528
mirrorFiles<\/code>\uff0c\u4f46\u662f\u8bb0\u5f55\u4e0b\u6765\u4e86libcuda.so<\/code> \u7684\u7248\u672c\u53f7\u548clibcuda-control.so<\/code> \u7684\u4f4d\u7f6e\u3002\u6ce8\u610f\uff0c\u8fd9\u4e2alibcuda-control<\/code> \u5c31\u662fvcuda-control<\/code> \u9879\u76ee\u751f\u6210\u7684\u7528\u6765\u62e6\u622acuda<\/code> \u8c03\u7528\u7684\u5e93\u3002<\/p>\n\u7136\u540e\u5c06
cudaControlFile<\/code> clone\u5230\u6240\u6709cudaSoname<\/code> \u548cmlSoName<\/code> \u4e2d\u5e93\u7684\u4f4d\u7f6e\u3002\u8fd9\u4e2a clone \u65b9\u6cd5\u4f1a\u5148\u5c1d\u8bd5\u786c\u94fe\u63a5\u8fc7\u53bb\uff0c\u5982\u679c\u5931\u8d25\u5c31\u76f4\u63a5\u590d\u5236\u8fc7\u53bb\u3002\u8fd9\u91cc\u7684cudaControlFile<\/code> \u5c31\u662f\u6211\u4eec\u4e0a\u9762\u6240\u8bf4\u7684libcuda-control.so<\/code> \u5566\u3002cudaSoname<\/code> \u548cmlSoName<\/code> \u5305\u542b\u4e86\u6240\u6709\u9700\u8981\u88ab\u62e6\u622a\u8c03\u7528\u7684\u5e93\u3002\u8fd9\u6837\u5b50\u5c31\u5b9e\u73b0\u4e86\u62e6\u622a\u6240\u6709\u7684cuda<\/code> \u8c03\u7528\u3002\u4e0b\u9762\u6211\u4eec\u5728\u770b\u4e00\u4e0bmirrorFiles<\/code> \u8fd9\u4e2a\u65b9\u6cd5\u5c31\u53ef\u4ee5\u4e86\u3002<\/p>\n\/\/ driver \u662f\u914d\u7f6e\u6587\u4ef6\u4e2d\u7684 \"nvidia\" \u6216 \"origin\"\n\/\/ vpath \u662f\u8981 mirror \u5230\u7684\u4f4d\u7f6e\uff0c\u5728 \/etc\/gpu-manager\/vdriver \u4e0b\u9762\nfunc (vm *VolumeManager) mirrorFiles(driver, vpath string, file string) error {\n \/\/ In computing, the Executable and Linkable Format (ELF, formerly named Extensible Linking Format), is a common standard file format for executable files, object code, shared libraries, and core dumps\n obj, err := elf.Open(file)\n defer obj.Close()\n\n \/\/ \u9ed1\u540d\u5355\u673a\u5236\uff0c\u5177\u4f53\u7528\u5904\u8fd8\u4e0d\u6e05\u695a\uff0c\u8ddf nvidia \u7684\u9a71\u52a8\u76f8\u5173\n ok, err := blacklisted(file, obj)\n if ok {\n return nil\n }\n l := path.Join(vpath, path.Base(file))\n \/\/ \u4e0d\u7ba1\u6709\u6ca1\u6709\uff0c\u5148\u5c1d\u8bd5\u628a gpu-manager \u91cc\u9762\u7684\u79fb\u9664\n if err := removeFile(l); err != nil {\n return err\n }\n \/\/ clone \u4f18\u5148\u786c\u8fde\u63a5\uff0c\u5176\u6b21\u662f\u590d\u5236\u6587\u4ef6\u5230\u6307\u5b9a\u4f4d\u7f6e\n if err := clone(file, l); err != nil {\n return err\n }\n \/\/ \u4ece elf \u4e2d\u83b7\u53d6\u5f53\u524d\u5e93\u7684 soname\n soname, err := obj.DynString(elf.DT_SONAME)\n if len(soname) > 0 {\n \/\/ \u5c06\u83b7\u53d6\u5230 soname \u7ec4\u6210\u8def\u5f84\n l = path.Join(vpath, soname[0])\n \/\/ \u5982\u679c\u6587\u4ef6\u548c\u5b83\u7684soname\u4e0d\u4e00\u81f4\uff08\u662f\u5426\u53ef\u4ee5\u8ba4\u4e3a\u8fd9\u4e2a\u6587\u4ef6\u662f\u8f6f\u94fe\u63a5\u8fc7\u53bb\u7684\uff09\n if err := linkIfNotSameName(path.Base(file), l); err != nil && !os.IsExist(err) {\n return err\n }\n\n \/\/ XXX Many applications (wrongly) assume that libcuda.so exists (e.g. with dlopen)\n \/\/ Hardcode the libcuda symlink for the time being.\n if strings.Contains(driver, \"nvidia\") {\n \/\/ \u8fd9\u91cc\u4e3a\u4ec0\u4e48\u8981\u79fb\u9664 libcuda.so \u548c libnvidia-ml.so \u7684\u8f6f\u94fe\u63a5\n \/\/ \u56e0\u4e3agpu\u8c03\u7528\u4f1a\u6d89\u53ca\u5230\u8fd9\u4e24\u4e2a\u5e93\uff0c\u8fd9\u4e24\u4e2a\u5e93\u4f1a\u8f6f\u94fe\u63a5\u5230\u771f\u5b9e\u7684\u5e93\u4e0a\u3002\u79fb\u9664\u540e\u66ff\u6362\u6210\u62e6\u622a\u7684\u5e93\n \/\/ Remove libcuda symbol link\n if vm.share && driver == \"nvidia\" && strings.HasPrefix(soname[0], \"libcuda.so\") {\n os.Remove(l)\n vm.cudaSoname[l] = l\n }\n\n \/\/ Remove libnvidia-ml symbol link\n if vm.share && driver == \"nvidia\" && strings.HasPrefix(soname[0], \"libnvidia-ml.so\") {\n os.Remove(l)\n vm.mlSoName[l] = l\n }\n\n \/\/ XXX GLVND requires this symlink for indirect GLX support\n \/\/ It won't be needed once we have an indirect GLX vendor neutral library.\n if strings.HasPrefix(soname[0], \"libGLX_nvidia\") {\n l = strings.Replace(l, \"GLX_nvidia\", \"GLX_indirect\", 1)\n if err := linkIfNotSameName(path.Base(file), l); err != nil && !os.IsExist(err) {\n return err\n }\n }\n }\n }\n\n return nil\n}\n<\/code><\/pre>\n\u8fd9\u6bb5\u4ee3\u7801\u4e2d\uff0c\u5148\u4f7f\u7528
blacklisted<\/code> \u6392\u9664\u4e00\u4e9b\u4e0d\u9700\u8981\u5904\u7406\u7684\u5e93\uff0c\u7136\u540e\u5c1d\u8bd5\u5c06\u5e93\u6216\u53ef\u6267\u884c\u6587\u4ef6 clone \u5230\u6211\u4eec\u7684\/etc\/gpu-manager\/vdriver<\/code> \u4e0b\u9762\u3002\/etc\/gpu-manager\/vdriver<\/code> \u4e0b\u9762\u6709\u4e24\u4e2a\u6587\u4ef6\u5939\uff0c\u4e00\u4e2a\u662fnvidia<\/code>\uff0c\u4fdd\u5b58\u4e86\u5df2\u7ecf\u88ab\u6211\u4eec\u62e6\u622a\u7684\u5e93\uff0c\u4e00\u4e2a\u662forigin<\/code>\uff0c\u8fd9\u91cc\u9762\u662f\u539f\u59cb\u7684\u672a\u5904\u7406\u7684\u5e93\u3002\u540c\u65f6\uff0c\u8fd8\u5c06 libcuda.so \u548c libnvidia-ml.so \u79fb\u9664\u4e86\uff0c\u8fd9\u6837\u5c31\u8c03\u7528\u4e0d\u5230\u771f\u5b9e\u7684\u5e93\u4e86\uff0c\u8f6c\u800c\u5728\u4e4b\u540e\u7528\u6211\u4eec\u62e6\u622a\u7684\u5e93\u6765\u66ff\u6362\u8fd9\u51e0\u4e2a\u6587\u4ef6\u3002<\/p>\n\u81f3\u6b64\uff0cvolumeManager \u5206\u6790\u7ed3\u675f\u3002<\/p>\n
gpu \u62d3\u6251\u7ed3\u6784\u611f\u77e5<\/h4>\n
- \n
![\"gpu](\"https:\/\/www.myway5.com\/wp-content\/uploads\/2020\/04\/Screenshot-from-2020-04-01-13-18-35.png\")
<\/p>\n![\"Device](\"https:\/\/www.myway5.com\/wp-content\/uploads\/2020\/03\/device-plugins.svg\")
<\/p>\n![\"gpu-manager-arch\"](\"https:\/\/www.myway5.com\/wp-content\/uploads\/2020\/04\/Screenshot-from-2020-04-01-13-57-48.png\")
<\/p>\n","protected":false},"excerpt":{"rendered":"