{"id":2838,"date":"2025-04-14T17:06:57","date_gmt":"2025-04-14T09:06:57","guid":{"rendered":"https:\/\/www.15zhi.net\/blog\/?p=2838"},"modified":"2025-04-14T17:06:57","modified_gmt":"2025-04-14T09:06:57","slug":"202504-%e8%ae%ba%e6%96%87%e7%a0%94%e8%af%bb-lowis3d-language-driven-open-world-instance-level-3d-scene-understanding","status":"publish","type":"post","link":"https:\/\/www.15zhi.net\/blog\/202504-%e8%ae%ba%e6%96%87%e7%a0%94%e8%af%bb-lowis3d-language-driven-open-world-instance-level-3d-scene-understanding\/","title":{"rendered":"202504 \u8bba\u6587\u7814\u8bfb-Lowis3D: Language-Driven Open-World Instance-Level 3D Scene Understanding"},"content":{"rendered":"\n

\u4f5c\u8005\uff1aRunyu Ding<\/a><\/a>;\u00a0Jihan Yang<\/a>;\u00a0Chuhui Xue<\/a><\/a>;
\u5355\u4f4d\uff1a\u9999\u6e2f\u5927\u5b66
\u6765\u6e90\uff1a\u00a0<\/strong>IEEE Transactions on Pattern Analysis and Machine Intelligence<\/a>
\u65f6\u95f4\uff1a2024.06.06<\/p>\n\n\n\n

\u4e00\u3001\u7814\u7a76\u80cc\u666f<\/strong><\/h2>\n\n\n\n

\u4e09\u7ef4\u573a\u666f\u8ba4\u77e5\u6d89\u53ca\u5b9a\u4f4d\u4e09\u7ef4\u7269\u4f53\u5e76\u7406\u89e3\u5176\u8bed\u4e49\uff0c\u662f\u865a\u62df\u73b0\u5b9e\uff08VR\uff09\u3001\u673a\u5668\u4eba\u64cd\u7eb5\u548c\u4eba\u673a\u4ea4\u4e92\u7b49\u73b0\u5b9e\u4e16\u754c\u5e94\u7528\u7684\u91cd\u8981\u611f\u77e5\u7ec4\u4ef6\u3002\u6df1\u5ea6\u5b66\u4e60\u5728\u8fd9\u4e00\u9886\u57df\u53d6\u5f97\u4e86\u663e\u8457\u6210\u5c31\uff0c\u7136\u800c\uff0c\u5728\u4eba\u7c7b\u6807\u6ce8\u6570\u636e\u96c6\u4e0a\u8bad\u7ec3\u7684\u6df1\u5ea6\u6a21\u578b\u53ea\u80fd\u7406\u89e3\u6570\u636e\u96c6\u4e2d\u5b58\u5728\u7684\u8bed\u4e49\u7c7b\u522b\uff0c\u800c\u65e0\u6cd5\u8bc6\u522b\u8bad\u7ec3\u6570\u636e\u4e2d\u672a\u51fa\u73b0\u7684\u65b0\u7c7b\u522b\uff0c\u8fd9\u4e25\u91cd\u9650\u5236\u4e86\u5b83\u4eec\u5728\u73b0\u5b9e\u4e16\u754c\u573a\u666f\u4e2d\u7684\u9002\u7528\u6027\uff0c\u4e14\u4e09\u7ef4\u6570\u636e\u96c6\u7684\u6ce8\u91ca\u6210\u672c\u5f88\u9ad8\u3002<\/p>\n\n\n\n

\u8fd9\u4fc3\u4f7f\u4f5c\u8005\u56e2\u961f\u7814\u7a76\u5f00\u653e\u4e16\u754c\u4e09\u7ef4\u5b9e\u4f8b\u7ea7\u573a\u666f\u7406\u89e3\uff0c\u5b83\u5141\u8bb8\u6a21\u578b\u8bc6\u522b\u548c\u5b9a\u4f4d\u672a\u5305\u542b\u5728\u6ce8\u91ca\u6570\u636e\u96c6\u6807\u7b7e\u7a7a\u95f4\u4e2d\u7684\u5f00\u653e\u96c6\u7c7b\u522b\u3002<\/p>\n\n\n\n

\u4e8c\u3001\u6838\u5fc3\u5185\u5bb9<\/strong><\/h2>\n\n\n\n
    \n
  • \u8d85\u7ecf\u9a8c\u7684\u8bed\u4e49\u8bc6\u522b\u548c\u5b9e\u4f8b\u5b9a\u4f4d\u4e24\u4e2a\u65b9\u9762\u6df1\u5165\u5206\u6790\u4e86\u5f00\u653e\u4e16\u754c\u4e09\u7ef4\u573a\u666f\u7406\u89e3\u6240\u9762\u4e34\u7684\u6311\u6218\uff0c\u6709\u52a9\u4e8e\u66f4\u597d\u5730\u7406\u89e3\u548c\u89e3\u51b3\u8fd9\u4e00\u4efb\u52a1\u3002<\/li>\n<\/ul>\n\n\n\n
      \n
    • \u63d0\u51fa\u4e86\u4e00\u79cd\u8f7b\u91cf\u7ea7\u63d0\u6848\u5206\u7ec4\u6a21\u5757\uff0c\u901a\u8fc7\u7ed3\u5408\u4f2a\u504f\u79fb\u76d1\u7763\u4fe1\u53f7\uff0c\u6709\u6548\u51cf\u5c11\u4e86\u5bf9\u57fa\u7840\u7c7b\u7684\u504f\u5dee\uff0c\u8fd9\u5927\u5927\u63d0\u9ad8\u4e86\u5b9e\u4f8b\u5b9a\u4f4d\u5bf9\u65b0\u7c7b\u522b\u7684\u9002\u5e94\u6027\u3002<\/li>\n<\/ul>\n\n\n\n
        \n
      • \u5728\u4e09\u4e2a\u5927\u89c4\u6a21\u573a\u666f\u7406\u89e3\u6570\u636e\u96c6\u4e0a\u8fdb\u884c\u4e86\u5e7f\u6cdb\u7684\u5b9e\u9a8c\uff0c\u8fd9\u4e9b\u6570\u636e\u96c6\u6db5\u76d6\u4e86\u5ba4\u5185\u548c\u5ba4\u5916\u573a\u666f\uff0c\u5728\u5b9e\u4f8b\u7ea7\u7406\u89e3\u65b9\u9762\u5927\u5927\u8d85\u8fc7\u4e86 PLA\u00a0\uff1b\u5728\u4e09\u7ef4\u5168\u666f\u5206\u5272\u4efb\u52a1\u4e0a\u8fdb\u4e00\u6b65\u5c1d\u8bd5 Lowis3D\uff0c\u5728 nuScenes \u6570\u636e\u96c6\u4e0a\u53d6\u5f97\u4e86\u663e\u8457\u6539\u8fdb\u3002<\/li>\n<\/ul>\n\n\n\n

        \u603b\u4e4b\uff0c\u8fd9\u4e9b\u6539\u8fdb\u4e3a\u5f00\u653e\u4e16\u754c\u4e09\u7ef4\u573a\u666f\u7406\u89e3\u63d0\u4f9b\u4e86\u4e00\u4e2a\u66f4\u5168\u9762\u3001\u66f4\u6709\u6548\u7684\u6846\u67b6\uff0c\u5728\u5404\u79cd\u73b0\u5b9e\u4e16\u754c\u573a\u666f\u4e2d\u5177\u6709\u5f88\u9ad8\u7684\u6f5c\u529b\u548c\u9002\u7528\u6027\u3002<\/p>\n\n\n\n

        \u4e09\u3001\u6838\u5fc3\u6846\u67b6\u8d21\u732e<\/strong><\/h2>\n\n\n\n
        \"\"<\/figure>\n\n\n\n

        \u4e0a\u9762\u662fLowis3d\u7684\u57f9\u8bad\u6846\u67b6\u63d2\u56fe\u3002Lowis3D \u4ee5\u56fe\u50cf\u6865\u63a5\u70b9\u8bed\u8a00\u5173\u8054\u6a21\u5757\u4e3a\u8d77\u70b9\uff0c\u8be5\u6a21\u5757\u53ef\u8de8\u5c42\u7ea7\u751f\u6210\u914d\u5bf9\u70b9\u4e91 p \u548c\u6807\u9898 t\uff0c\u4ece\u800c\u4f7f\u6a21\u578b\u80fd\u591f\u901a\u8fc7\u5bf9\u6bd4\u8bad\u7ec3 Lcap \u5b66\u4e60\u4e30\u5bcc\u7684\u8bed\u4e49\u3002 \u6b64\u5916\uff0c\u4e3a\u4e86\u6821\u51c6\u6a21\u578b\u5bf9\u57fa\u7840\u8bed\u4e49\u7684\u504f\u5411\uff0c\u4f5c\u8005\u52a0\u5165\u4e86\u4e00\u4e2a\u4e8c\u5143\u5934 Fb\uff0c\u901a\u8fc7 Lbi \u9884\u6d4b\u5c5e\u4e8e\u5df2\u89c1\u6216\u672a\u89c1\u7c7b\u522b\u7684\u70b9\uff0c\u8be6\u89c1\u7b2c V-B \u8282\u3002\u6b64\u5916\uff0c\u5bf9\u4e8e\u65b0\u5b9e\u4f8b\u5b9a\u4f4d\uff0c\u53bb\u504f\u5b9e\u4f8b\u5b9a\u4f4d\u6a21\u5757\u901a\u8fc7 Loff \u5bf9\u65b0\u7c7b\u522b\u751f\u6210\u53ef\u4fe1\u7684\u4f2a\u504f\u79fb\u76d1\u7763\uff0c\u4ee5\u589e\u5f3a\u5f00\u653e\u4e16\u754c\u5bf9\u8c61\u6027\u5b66\u4e60<\/p>\n\n\n\n

        \"\"<\/figure>\n\n\n\n

        \u4e0a\u56fe\u663e\u793a\u7684\u662f\u56fe\u50cf\u6865\u63a5\u70b9\u8bed\u8a00\u5173\u8054\u3002\u4f5c\u8005\u56e2\u961f\u63d0\u51fa\u4e86\u573a\u666f\u7ea7\u3001\u89c6\u56fe\u7ea7\u548c\u5b9e\u4f53\u7ea7\u7684\u5206\u5c42\u70b9\u8bed\u8a00\u5173\u8054\u65b9\u6cd5\uff0c\u901a\u8fc7\u89c6\u89c9\u8bed\u8a00\u57fa\u7840\u6a21\u578b\u548c\u591a\u89c6\u56fe RGB \u56fe\u50cf\uff0c\u5728\u6807\u9898\u76d1\u7763\u4e0b\u5206\u914d\u4ece\u7c97\u5230\u7ec6\u7684\u70b9\u96c6\u3002<\/p>\n\n\n\n

        \u56db\u3001\u5b9e\u9a8c\u90e8\u5206<\/strong><\/h2>\n\n\n\n
          \n
        • \u6570\u636e\u96c6\uff1a \u4e3a\u4e86\u5168\u9762\u9a8c\u8bc1Lowis3D\u7684\u6709\u6548\u6027\uff0c\u4f5c\u8005\u5728\u4e24\u4e2a\u5ba4\u5185\u6570\u636e\u96c6\u4e0a\u8fdb\u884c\u4e86\u8bed\u4e49\u548c\u5b9e\u4f8b\u5206\u5272\u4efb\u52a1\u7684\u5b9e\u9a8c\uff0c\u8fd9\u4e24\u4e2a\u6570\u636e\u96c6\u5206\u522b\u662fScanNet \uff08\u6ce8\u91ca\u4e3a20\u4e2a\u7c7b\u522b\uff09\u548cS3DIS \uff08\u6ce8\u91ca\u4e3a13\u4e2a\u7c7b\u522b\uff09\u3002\u6b64\u5916\u8fd8\u5728\u5ba4\u5916\u6570\u636e\u96c6 nuScenes \u4e0a\u5bf9 Lowis3D \u8fdb\u884c\u4e86\u8bc4\u4f30\uff0c\u8be5\u6570\u636e\u96c6\u5305\u542b 16 \u4e2a\u5168\u666f\u5206\u5272\u7c7b\u522b\u3002<\/li>\n\n\n\n
        • \u7f51\u7edc\u67b6\u6784\uff1a \u4f5c\u8005\u91c7\u7528\u6d41\u884c\u7684\u9ad8\u6027\u80fd\u7a00\u758f\u5377\u79ef UNet \u4f5c\u4e3a\u4e09\u7ef4\u7f16\u7801\u5668 F3D\uff0c\u91c7\u7528 CLIP \u7684\u6587\u672c\u7f16\u7801\u5668\u4f5c\u4e3a Ft\uff0c\u91c7\u7528\u6279\u91cf\u5f52\u4e00\u5316\u7684\u5168\u8fde\u63a5\u5c42 \u548c ReLU \u4f5c\u4e3a VL \u9002\u914d\u5668 F\u03b8\uff0c\u91c7\u7528 UNet \u89e3\u7801\u5668\u4f5c\u4e3a\u4e8c\u8fdb\u5236\u5934 Fb\u3002\u6b64\u5916\u8fd8\u91c7\u7528\u4e86\u6700\u5148\u8fdb\u7684\u5b9e\u4f8b\u5206\u5272\u7f51\u7edc SoftGroup \uff0c\u7528\u4e8e\u63d0\u6848\u5206\u7ec4 Foff \u548c\u8bc4\u5206 Fscr\u3002<\/li>\n\n\n\n
        • \u57fa\u51c6\u65b9\u6cd5\uff1a \u5bf9\u4e8e\u5b9e\u4f8b\u548c\u5168\u666f\u5206\u5272\uff0c\u4f5c\u8005\u91c7\u7528 OV-Softgroup \u4f5c\u4e3a\u57fa\u51c6\u65b9\u6cd5\u3002\u9274\u4e8e\u5b9e\u4f8b\u7ea7\u5f00\u653e\u4e16\u754c\u4e09\u7ef4\u573a\u666f\u7406\u89e3\u4ecd\u5904\u4e8e\u8d77\u6b65\u9636\u6bb5\uff0c\u76ee\u524d\u8fd8\u6ca1\u6709\u5176\u4ed6\u5408\u9002\u7684\u65b9\u6cd5\u53ef\u4f9b\u76f4\u63a5\u6bd4\u8f83\u3002<\/li>\n<\/ul>\n\n\n\n

          \u4e0b\u9762\u662f\u5bf9\u5e94\u4e0d\u540c\u6570\u636e\u96c6\u4e0a\u5404\u4e2a\u6a21\u578b\u7684\u8868\u73b0\uff0c\u6570\u636e\u8bb0\u5f55\u5982\u4e0b<\/p>\n\n\n\n

          \"\"<\/figure>\n\n\n\n

          \u4e0b\u56fe\u662f\u8bc6\u522b\u8bcd\u6c47\u5916\u7c7b\u522b\u7684\u5b9a\u6027\u793a\u4f8b\u3002(a) \u663e\u793a\u540c\u4e49\u8bcd\u7c7b\u522b\u7684\u8bc6\u522b\u7ed3\u679c\u3002(b) \u5c55\u793a\u62bd\u8c61\u6982\u5ff5\u7684\u5206\u7c7b\u7ed3\u679c\u3002(c) \u5c55\u793a\u672a\u6ce8\u91ca\u7c7b\u522b\u7684\u5206\u7c7b\u7ed3\u679c\u3002<\/p>\n\n\n\n

          \"\"<\/figure>\n\n\n\n

          \u4e0b\u56fe\u662f\u5f00\u653e\u4e16\u754c\u5b9e\u4f8b\u548c\u6cdb\u89c6\u5206\u5272\u7684\u5b9a\u6027\u7ed3\u679c\u3002\u65b0\u7c7b\u522b\u7528\u5f69\u8272\u6807\u6ce8\uff0c\u57fa\u7840\u7c7b\u522b\u7528\u7070\u8272\u6807\u6ce8\uff0c\u4ee5\u4fbf\u6e05\u6670\u533a\u5206\u3002\u503c\u5f97\u6ce8\u610f\u7684\u6bd4\u8f83\u5728\u7ea2\u8272\u8fb9\u754c\u6846\u5185\u7a81\u51fa\u663e\u793a\u3002<\/p>\n\n\n\n

          \"\"<\/figure>\n\n\n\n

          \u4e94\u3001\u603b\u7ed3<\/strong><\/h2>\n\n\n\n

          \u4f5c\u8005\u63d0\u51fa\u7684 Lowis3D \u662f\u4e00\u4e2a\u5168\u9762\u800c\u9ad8\u6548\u7684\u6846\u67b6\uff0c\u7528\u4e8e\u89e3\u51b3\u5f00\u653e\u4e16\u754c\u5b9e\u4f8b\u7ea7\u4e09\u7ef4\u573a\u666f\u7406\u89e3\u95ee\u9898\u3002\u65b9\u6cd5\u662f\u5229\u7528\u56fe\u50cf\u4f5c\u4e3a\u6865\u6881\uff0c\u5efa\u7acb\u5206\u5c42\u7684\u70b9-\u5b57\u5e55\u5bf9\uff0c\u5229\u7528\u4e8c\u7ef4\u89c6\u89c9\u8bed\u8a00\uff08VL\uff09\u57fa\u7840\u6a21\u578b\u7684\u529b\u91cf\u4ee5\u53ca\u4e09\u7ef4\u573a\u666f\u548c\u4e8c\u7ef4\u56fe\u50cf\u4e4b\u95f4\u7684\u51e0\u4f55\u5173\u7cfb\u3002\u5bf9\u6bd4\u5b66\u4e60\u88ab\u7528\u6765\u589e\u5f3a\u8fd9\u4e9b\u5173\u8054\u5bf9\u4e2d\u7684\u7279\u5f81\u5bf9\u9f50\uff0c\u4ece\u800c\u4e3a\u4e09\u7ef4\u7f51\u7edc\u6ce8\u5165\u4e86\u4e30\u5bcc\u7684\u8bed\u4e49\u6982\u5ff5\u3002\u6b64\u5916\u8fd8\u63d0\u51fa\u4e86\u504f\u79bb\u5b9e\u4f8b\u5b9a\u4f4d\u7684\u65b9\u6cd5\uff0c\u4ee5\u51cf\u8f7b\u5bf9\u8c61\u5206\u7ec4\u5bf9\u57fa\u672c\u6a21\u5f0f\u7684\u504f\u5dee\uff0c\u4ece\u800c\u63d0\u9ad8\u5bf9\u8c61\u6027\u5b66\u4e60\u7684\u6cdb\u5316\u80fd\u529b\u3002\u5927\u91cf\u5b9e\u9a8c\u8bc1\u660e\u4e86\u4f5c\u8005\u7684\u65b9\u6cd5\u5728\u5f00\u653e\u4e16\u754c\u5b9e\u4f8b\u7ea7\u573a\u666f\u7406\u89e3\u4efb\u52a1\u4e2d\u7684\u6709\u6548\u6027\u3002<\/p>\n\n\n\n

          <\/p>\n","protected":false},"excerpt":{"rendered":"

          \u4f5c\u8005\uff1aRunyu Ding;\u00a0Jihan Yang;\u00a0Chuhui Xue;\u5355\u4f4d\uff1a\u9999\u6e2f\u5927\u5b66\u6765\u6e90\uff1a\u00a0IEEE T […]<\/p>\n","protected":false},"author":63,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-2838","post","type-post","status-publish","format-standard","hentry","category-events"],"_links":{"self":[{"href":"https:\/\/www.15zhi.net\/blog\/wp-json\/wp\/v2\/posts\/2838","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.15zhi.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.15zhi.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.15zhi.net\/blog\/wp-json\/wp\/v2\/users\/63"}],"replies":[{"embeddable":true,"href":"https:\/\/www.15zhi.net\/blog\/wp-json\/wp\/v2\/comments?post=2838"}],"version-history":[{"count":2,"href":"https:\/\/www.15zhi.net\/blog\/wp-json\/wp\/v2\/posts\/2838\/revisions"}],"predecessor-version":[{"id":2847,"href":"https:\/\/www.15zhi.net\/blog\/wp-json\/wp\/v2\/posts\/2838\/revisions\/2847"}],"wp:attachment":[{"href":"https:\/\/www.15zhi.net\/blog\/wp-json\/wp\/v2\/media?parent=2838"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.15zhi.net\/blog\/wp-json\/wp\/v2\/categories?post=2838"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.15zhi.net\/blog\/wp-json\/wp\/v2\/tags?post=2838"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}