We use cookies to ensure you get the best experience on our website
首頁 探索 說明
註冊 登入
bohanleng
/
SheetAssemblyDT
Wiki
頁面: SheetAssemblyDT release v1.0
SheetAssemblyDT release v1.0
SheetAssemblyDT
SheetAssemblyDT release v1.0
bohanleng 於 1 年之前 修改了此頁面

SheetAssemblyDT

The Unreal Engine 5 dev repository for the Digital Twin project at the Metal Sheet Assembly Shop-floor, Zoomlion.

Developed by Bohan LENG

Thumbnail

Table of Contents
  1. About The Repository
  2. Features
  3. Code Overview

About the Repository

This repository contains the Unreal Engine project for the Digital Twin platform designed for the Metal Sheet Assembly Shop-floor, focusing on the warehouse and logistics systems. As illustrated in the architecture diagram below, this repository specifically covers the Unreal Engine section.

The communication infrastructure and related components are managed separately in the ZMonitor repository.

---
title: Digital Twin Implementation Architecture
---
flowchart BT
    
    ue["`***Unreal Engine***`"]
    web[Web Client]
    proxy[HTTP Proxy]
    iot[IoT, <br>Internet of Things]
    rcs[RCS, <br>Robot Control System]
    wcs[WCS, <br>Warehouse Control System]
    agv[AGV, <br>Automated Guide Vehicle]
    dev[Shuttle, Hoist, Conveyor]
    asrs[ASRS, <br>Automated Storage and Retrieval System]
    wms[WMS, <br>Warehouse Management System]


    ue<-- "`***Web Interface***`" -->web
    proxy -- TCP --> ue
    iot -- HTTP --> proxy
    rcs -- HTTP --> iot
    wcs -- HTTP --> iot
    agv --> rcs
    dev --> wcs
    asrs --> wms
    wms -- HTTP --> proxy

Class Diagram

---
title: Class Diagram of SheetMetalAssemblyDT
---

classDiagram

    class TwinningEntity {
    	<<interface>>
			Panel info_panel
			+ OnClick()
			+ Update()*
			+ CalculateFocusTarget()*
			+ MakeOnSelectedWebInterfaceData()*
    }
    
    class Vehicle {
    	<<interface>>
    	+ DisplayPath()
    }

    class AGV {
			+ static ConvertCoords(x, y, angle) x, y, angle
    }

    class Shuttle {

    }

    class Hoist {
		 
    }

    class Storage {
    	+ ConfigInfo config_info
    	+ StorageInfo[] storage_infos
			+ IsEmptyContainer() bool
			+ HighlightStorage()
    }

    class TwinningEntityManager {
      - map~id, agv~ agv_dict
      - map~id, shuttle~ shuttle_dict
      - map~id, hoist~ hoist_dict
      
      + RegisterTwinningEntity(TwinningEntity te)
      + GetTwinningEntity(type, id) TwinningEntity
      + GetAllTwinningEntities() TwinningEntity[]
      + GetAllDevices() TwinningEntity[]
      + OnAGVUpdate(update)
      + OnShuttleUpdate(update)
      + OnHoistUpdate(update)
    }
	
	
	
	class StorageManager {
		- map~IntVector3, BinConfig~ bin_config_dict
		+ GetBinName(coord) string
		+ GetBinCoord(name) IntVector3
		+ GetBinLocation(coord) Vector3
		+ RecalculateAllBinLocation()
		+ FilterFromToBins(from_coord, to_coord) BinID[]
		+ OnStorageUpdate(update)
		+ HighlightFilteredBins(bin_ids)
	}
	
    
	class CMonitorSPI {
      <<interface>>
      + OnConnected()*
      + OnRspAuth(auth_msg)*
      + OnDisconnected()*
      + On...Update()*
  }
	
	class MonitorAPIUser {
		+ TwinningEntityManager twinning_entity_manager_ref
		+ OnAGVUpdate(update)
		+ OnShuttleUpdate(update)
		+ HoistUpdate(update)
		+ OnStorageUpdate(update)
	}
	
	class MonitorAPIManager {
		+ ConfigManager config_manager_ref
		- MonitorAPIUser monitor_api_user_ptr
		+ ConnectAPIUser()
	}

	
	class ConfigManager {
		+ string web_url
		+ string http_proxy_ip
		+ int http_proxy_port
		+ string subscriber_id
		+ int graphics_quality
	}

TwinningEntity <|-- Vehicle	
Vehicle <|.. AGV	
Vehicle <|.. Shuttle	
TwinningEntity <|-- Hoist
TwinningEntity <|-- Storage

AGV o-- TwinningEntityManager: n
Shuttle o-- TwinningEntityManager: 8
Hoist o-- TwinningEntityManager: 4
StorageManager o-- TwinningEntityManager: 1
Storage o-- StorageManager: n

TwinningEntityManager o-- MonitorAPIUser: 
StorageManager o-- MonitorAPIUser: 

MonitorAPIUser o-- MonitorAPIManager
ConfigManager o-- MonitorAPIManager

CMonitorSPI <|.. MonitorAPIUser

This class diagram is illustrative. Every class is instead implemented as a mixture of UE C++ and Blueprint.

Features

Embedded Web interface

In Zoomlion there are mature workflow of building Digital Twin applicaitons using Web. The data interfaces of different equipment providers could be easily integrated into Web and provide clear data statistics. However in terms of 3D rendering, Web can hardly support such great amount of entities in factory, for each entity should be presented with a complex geometry to be presented.

At Zoomlion, there are mature workflows for building Digital Twin applications using web technologies. Web-based data interfaces from various equipment providers enable seamless integration and clear data visualization. However, in the context of 3D rendering, web technologies face challenges in handling the large number of entities in a factory, as each entity must be represented by complex geometries.

The Plugins/WebUI module was developed to integrate web-based kanban functionality into Unreal Engine, leveraging the strengths of both technologies. In this project, the UBrowserWidget and AWebManager classes are implemented, extending the UWebInterface class. The method UWebInterface::LoadURL() is used to load the pre-designed web interface, while interactions between the web interface and the Digital Twin scene are managed through AWebManager::OnWebInterface. Behaviors are defined for the following events:

  • onEquioSelectedFromWeb
  • showEquipLabel
  • hideEquipLabel
  • onStocksFiltered
  • onLogisticsTaskSelected
  • onSceneChanged

Shop-floor Environment Building

In addition to modeling ASRS (Automated Storage and Retrieval Systems) and associated devices, this project includes high-precision models of the entire assembly shop-floor environment.

environment1

environment2

environment3

Device management and real-time info panel

The Device Management tab lists all connected devices, displaying real-time information from their respective systems. Double-clicking a device entry focuses the viewport on the device’s location in the Digital Twin environment.

device_locationing

Each device is interactive and clickable. Clicking on a device focuses the viewport on it. For vehicles (e.g., AGVs, shuttles), an additional display shows their destinations upon selection.

Real-time info panels are displayed above AGVs, shuttles, hoists, and hoist conveyors. These panels show live status updates and errors. If an error occurs, the device is highlighted with blinking red, and the panel displays the error details.

device_panel

Storage Management

The ASRS on the shop floor contains over 3,000 storage bins, each with trays and one or more materials stored. This project provides an intuitive and comprehensive visualization of the ASRS and its bins.

storage

Each bin is represented by a tray, a general box (if loaded), and a levitating panel. The panel displays the bin’s ID, materials stored, and their quantities. By default, the panels are not rendered. Only bins within a 6-meter distance from the spectator (defined in AShopfloorSpectatorPawn) are displayed.

Clicking a bin focuses the camera on it and brings up a detailed web menu. Searching for bins is supported through buttons like “same material,” “same batch,” or “empty container.” Search results are highlighted and displayed in both the 3D scene and the web interface.

storage_managment

Logistics Management

logistics_management

In the Logistics Management panel, logistic tasks can be located by clicking on orders in the web panel. Devices associated with a specific task are filtered, highlighted, and focused in the scene.

Storage bins configuration

For the UE generated storage bins of the ASRS, there are some to be ruled out because the location may be unavailable (next to the hoisters, in the air, etc.). A configuration feature was built in StorageManager to manually set the availability of each bin. Multiple selection is supported to facilitate this process.

warehouse_config

Code Overview

  28 .gitignore
   2 Config/DefaultEditor.ini
 154 Config/DefaultEngine.ini
  20 Config/DefaultGame.ini
  85 Config/DefaultInput.ini
 161 Config/amr_map_reduced.json
   9 Config/runtime_config.json
3521 Config/storage_config.csv *
 358 README.md
 294 SheetAssemblyDT.png
  33 SheetAssemblyDT.uproject
  15 Source/SheetAssemblyDT.Target.cs
  35 Source/SheetAssemblyDT/Private/AGV.cpp
  26 Source/SheetAssemblyDT/Private/AGVManager.cpp
  66 Source/SheetAssemblyDT/Private/AMRAPIComm.cpp
   5 Source/SheetAssemblyDT/Private/BaseTEPanel.cpp
  40 Source/SheetAssemblyDT/Private/BrowserWidget.cpp
  44 Source/SheetAssemblyDT/Private/ConfigManager.cpp
  16 Source/SheetAssemblyDT/Private/DataProcessingHelpler.cpp
  86 Source/SheetAssemblyDT/Private/ExpDataSender.cpp
  34 Source/SheetAssemblyDT/Private/FileOperationHelper.cpp
  32 Source/SheetAssemblyDT/Private/Hoist.cpp
  27 Source/SheetAssemblyDT/Private/HoistManager.cpp
  63 Source/SheetAssemblyDT/Private/MecanumAGV.cpp
  69 Source/SheetAssemblyDT/Private/MonitorAPIManager.cpp
 308 Source/SheetAssemblyDT/Private/MonitorAPIUser.cpp
  29 Source/SheetAssemblyDT/Private/RGVManager.cpp
  13 Source/SheetAssemblyDT/Private/ShopfloorSpectatorPawn.cpp
  21 Source/SheetAssemblyDT/Private/Shuttle.cpp
  22 Source/SheetAssemblyDT/Private/Storage.cpp
 224 Source/SheetAssemblyDT/Private/StorageManager.cpp
 152 Source/SheetAssemblyDT/Private/TwinningEntity.cpp
  84 Source/SheetAssemblyDT/Private/TwinningEntityManager.cpp
   5 Source/SheetAssemblyDT/Private/Vehicle.cpp
  52 Source/SheetAssemblyDT/Private/ViewportManager.cpp
 118 Source/SheetAssemblyDT/Private/WebManager.cpp
  30 Source/SheetAssemblyDT/Public/AGV.h
  43 Source/SheetAssemblyDT/Public/AGVManager.h
  37 Source/SheetAssemblyDT/Public/AMRAPIComm.h
  23 Source/SheetAssemblyDT/Public/BaseTEPanel.h
  32 Source/SheetAssemblyDT/Public/BrowserWidget.h
  48 Source/SheetAssemblyDT/Public/ConfigManager.h
  20 Source/SheetAssemblyDT/Public/DataProcessingHelper.h
  57 Source/SheetAssemblyDT/Public/ExpDataSender.h
  21 Source/SheetAssemblyDT/Public/FileOperationHelper.h
  31 Source/SheetAssemblyDT/Public/Hoist.h
  38 Source/SheetAssemblyDT/Public/HoistManager.h
  39 Source/SheetAssemblyDT/Public/MecanumAGV.h
  49 Source/SheetAssemblyDT/Public/MonitorAPIManager.h
  94 Source/SheetAssemblyDT/Public/MonitorAPIUser.h
  39 Source/SheetAssemblyDT/Public/RGVManager.h
 272 Source/SheetAssemblyDT/Public/SheetMetalAssemblyUEDef.h
  19 Source/SheetAssemblyDT/Public/ShopfloorSpectatorPawn.h
  26 Source/SheetAssemblyDT/Public/Shuttle.h
  67 Source/SheetAssemblyDT/Public/Storage.h
 143 Source/SheetAssemblyDT/Public/StorageManager.h
 126 Source/SheetAssemblyDT/Public/TwinningEntity.h
  95 Source/SheetAssemblyDT/Public/TwinningEntityManager.h
  21 Source/SheetAssemblyDT/Public/Vehicle.h
  38 Source/SheetAssemblyDT/Public/ViewportManager.h
  70 Source/SheetAssemblyDT/Public/WebManager.h
  29 Source/SheetAssemblyDT/Public/ZoomlionDef.h
 101 Source/SheetAssemblyDT/SheetAssemblyDT.Build.cs
   6 Source/SheetAssemblyDT/SheetAssemblyDT.cpp
   6 Source/SheetAssemblyDT/SheetAssemblyDT.h
   5 Source/SheetAssemblyDT/SheetAssemblyDTGameModeBase.cpp
  17 Source/SheetAssemblyDT/SheetAssemblyDTGameModeBase.h
  15 Source/SheetAssemblyDTEditor.Target.cs
  10 uecleanproj.sh
4397 total

source of release v1.0, * config file not accounting for total lines. 172 MB binaries in Content, Plugins/WebUI, ThirdParties/nlohmann_json are not included either.

刪除頁面

這將刪除頁面 "SheetAssemblyDT release v1.0"。請三思而後行。

取消操作
確認操作