pytorch

b5a5e296 - Support 3D mesh/point cloud (#20413)

Support 3D mesh/point cloud (#20413) Summary: I started adding support for the new **[mesh/point cloud](https://github.com/tensorflow/graphics/blob/master/tensorflow_graphics/g3doc/tensorboard.md)** data type introduced to TensorBoard recently. I created the functions to add the data, created the appropriate summaries. This new data type however requires a **Merged** summary containing the data for the vertices, colors and faces. I got stuck at this stage. Maybe someone can help. lanpa? I converted the example code by Google to PyTorch: ```python import numpy as np import trimesh import torch from torch.utils.tensorboard import SummaryWriter sample_mesh = 'https://storage.googleapis.com/tensorflow-graphics/tensorboard/test_data/ShortDance07_a175_00001.ply' log_dir = 'runs/torch' batch_size = 1 # Camera and scene configuration. config_dict = { 'camera': {'cls': 'PerspectiveCamera', 'fov': 75}, 'lights': [ { 'cls': 'AmbientLight', 'color': '#ffffff', 'intensity': 0.75, }, { 'cls': 'DirectionalLight', 'color': '#ffffff', 'intensity': 0.75, 'position': [0, -1, 2], }], 'material': { 'cls': 'MeshStandardMaterial', 'roughness': 1, 'metalness': 0 } } # Read all sample PLY files. mesh = trimesh.load_remote(sample_mesh) vertices = np.array(mesh.vertices) # Currently only supports RGB colors. colors = np.array(mesh.visual.vertex_colors[:, :3]) faces = np.array(mesh.faces) # Add batch dimension, so our data will be of shape BxNxC. vertices = np.expand_dims(vertices, 0) colors = np.expand_dims(colors, 0) faces = np.expand_dims(faces, 0) # Create data placeholders of the same shape as data itself. vertices_tensor = torch.as_tensor(vertices) faces_tensor = torch.as_tensor(faces) colors_tensor = torch.as_tensor(colors) writer = SummaryWriter(log_dir) writer.add_mesh('mesh_color_tensor', vertices=vertices_tensor, faces=faces_tensor, colors=colors_tensor, config_dict=config_dict) writer.close() ``` I tried adding only the vertex summary, hence the others are supposed to be optional. I got the following error from TensorBoard and it also didn't display the points: ``` Traceback (most recent call last): File "/home/dawars/workspace/pytorch/venv/lib/python3.6/site-packages/werkzeug/serving.py", line 302, in run_wsgi execute(self.server.app) File "/home/dawars/workspace/pytorch/venv/lib/python3.6/site-packages/werkzeug/serving.py", line 290, in execute application_iter = app(environ, start_response) File "/home/dawars/workspace/pytorch/venv/lib/python3.6/site-packages/tensorboard/backend/application.py", line 309, in __call__ return self.data_applications[clean_path](environ, start_response) File "/home/dawars/workspace/pytorch/venv/lib/python3.6/site-packages/werkzeug/wrappers/base_request.py", line 235, in application resp = f(*args[:-2] + (request,)) File "/home/dawars/workspace/pytorch/venv/lib/python3.6/site-packages/tensorboard/plugins/mesh/mesh_plugin.py", line 252, in _serve_mesh_metadata tensor_events = self._collect_tensor_events(request) File "/home/dawars/workspace/pytorch/venv/lib/python3.6/site-packages/tensorboard/plugins/mesh/mesh_plugin.py", line 188, in _collect_tensor_events tensors = self._multiplexer.Tensors(run, instance_tag) File "/home/dawars/workspace/pytorch/venv/lib/python3.6/site-packages/tensorboard/backend/event_processing/plugin_event_multiplexer.py", line 400, in Tensors return accumulator.Tensors(tag) File "/home/dawars/workspace/pytorch/venv/lib/python3.6/site-packages/tensorboard/backend/event_processing/plugin_event_accumulator.py", line 437, in Tensors return self.tensors_by_tag[tag].Items(_TENSOR_RESERVOIR_KEY) KeyError: 'mesh_color_tensor_COLOR' ``` Pull Request resolved: https://github.com/pytorch/pytorch/pull/20413 Differential Revision: D15500737 Pulled By: orionr fbshipit-source-id: 426e8b966037d08c065bce5198fd485fd80a2b67