Recently, the Helmholtz-wide software development platform (Gitlab) has been extended with the ability to create diagrams from textual descriptions. This post will help you getting started with this new feature.
What is it about and where can it be applied?
Gitlab is perfect for colloborative work on codes, scripts and other shared files. Now you can also create virtually all sorts of diagrams and sketches directly in Gitlab, both allowing to quickly create plots, as well as collaboratively build and maintain complex graphs.
All graphs are built from simple scripts directly in Gitlab using a plugin named Kroki. Such scripts can be used in any place on the Helmholtz-wide Gitlab where Markdown is supported:
- Markdown documents inside repositories
- Comments
- Issues
- Merge requests
- Milestones
- Snippets (the snippet must be named with a
.mdextension) - Wiki pages
Click here for examples in Gitlab!
Quick-Start Example
Let us jump right in with an example by creating a sequence diagram, using the common Unified Modeling Language (UML).
Put this content describing the diagram in any of the items listed above.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
```plantuml
@startuml
participant User
User -> A: DoWork
activate A
A -> B: << createRequest >>
activate B
B -> C: DoWork
activate C
C --> B: WorkDone
destroy C
B --> A: RequestCreated
deactivate B
A -> User: Done
deactivate A
@enduml
```
This description is then converted into this sequence diagram.
How Does That Work?
In the background a tool called Kroki is doing the magic. It takes the textual description and converts it into an image. The example shown above is using PlantUML to create the diagram of our choice (see line 1).
So all you need to do is to create the textual description of your diagram and put it into GitLab Markdown. When viewing the content, in the background we will make sure that you are presented an image of your diagram description.
Step-By-Step Guide
In this example our goal is to create a visual representation of JSON data. Let’s take this example JSON data.
Show JSON data
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
{
"firstName": "John",
"lastName": "Smith",
"age": 28,
"address": {
"streetAddress": "Bautzner Landstr. 400",
"city": "Dresden",
"state": "Saxony",
"postalCode": "01328"
},
"offices": [
{
"building": 512,
"room": 1234
},
{
"building": 312,
"room": 3
}
],
"departments": []
}
We will use PlantUML to display the JSON data.
At first, we need to tell GitLab that we want to create a diagram using PlantUML. This is why we create a Markdown code block using PlantUML for syntax highlighting.
1
2
3
```plantuml
```
The PlantUML documentation tells us how such a
visual representation can be created.
In this case you need to embed the JSON data into a block starting with
@startjson and ending with @endjson.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
```plantuml
@startjson
{
"firstName": "John",
"lastName": "Smith",
"age": 28,
"address": {
"streetAddress": "Bautzner Landstr. 400",
"city": "Dresden",
"state": "Saxony",
"postalCode": "01328"
},
"offices": [
{
"building": 512,
"room": 1234
},
{
"building": 312,
"room": 3
}
],
"departments": []
}
@endjson
```
You will be presented an image as shown below.
Gantt Diagram
PlantUML also supports Gantt diagrams:
Show code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
```plantuml
@startgantt
printscale weekly
Project starts the 1st of July 2021
[Literature Review] lasts 4 weeks
[Literature Review] is 100% completed
then [Run Experiments] lasts 12 weeks
[Run Experiments] is 70% completed
-- Phase Two --
then [Data Analysis] lasts 8 weeks
[Data Analysis] is 0% completed
then [Write Publication] lasts 6 weeks
[Write Publication] is 0% completed
@endgantt
```
More in-depth information on the Gantt diagram syntax is available in the official PlantUML documentation.
What Else Can Be Done?
The new feature is not limited to using PlantUML. The new integration gives you access to more than that. It supports, among others, Vega, C4 with PlantUML, GraphViz or BlockDiag. For a complete list please refer to the official list of supported diagram types and formats.
Vega
Word Cloud
The following wordcloud is generated from the mission statement of the Helmholtz Association using Vega.
Show code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
```vega
{
"$schema": "https://vega.github.io/schema/vega/v5.json",
"description": "A word cloud visualization depicting the mission statement of Helmholtz.",
"width": 800,
"height": 400,
"padding": 0,
"data": [
{
"name": "table",
"values": [
"We contribute to solving the major challenges facing society, science and the economy by conducting top-level research in strategic programmes within our six research fields: Energy, Earth & Environment, Health, Aeronautics, Space and Transport, Matter, and Information. We research highly complex systems using our large-scale devices and infrastructure, cooperating closely with national and international partners. We contribute to shaping our future by combining research and technology development with perspectives for innovative application and provisions in tomorrow's world. We attract and promote the best young talents, offering a unique research environment and general support throughout all career stages.",
"The Helmholtz Association performs cutting-edge research which contributes substantially to solving the grand challenges of science, society and industry. Helmholtz Association scientists focus on researching the highly-complex systems which determine human life and the environment, for example: ensuring that society remains mobile, ensuring that society has a reliable energy supply, ensuring that future generations live in a secure ecosystem, and developing treatments for previously incurable diseases.",
"The Helmholtz Association performs cutting-edge research which contributes substantially to solving the grand challenges of science, society and industry. Helmholtz Association scientists focus on researching the highly-complex systems which determine human life and the environment, for example: ensuring that society remains mobile, ensuring that society has a reliable energy supply, ensuring that future generations live in a secure ecosystem, and developing treatments for previously incurable diseases.",
"The activities of the Helmholtz Association focus on securing the long-term foundations of human life and on creating the technological basis for a competitive economy. Our potential, as an Association, to achieve these goals is due to the outstanding scientists working at our 18 major research centres, a high-performance infrastructure and modern research management.",
"Our scientists develop national research programmes for each of these fields and international experts review these programmes. These expert evaluations form the basis of the programme-oriented funding (POF) which finances to Helmholtz Association research.",
"Within the six research fields, Helmholtz scientists cooperate with each other and with external partners - working across disciplines, organisations and national borders. Indeed, the being part of the Helmholtz Association means making concerted research effort in which networks form the key to inquiring thought and action. Concerted research is efficient and flexible. The Helmholtz Association uses this research to create an effective basis for shaping the future."
],
"transform": [
{
"type": "countpattern",
"field": "data",
"case": "upper",
"pattern": "[\\w']{3,}",
"stopwords": "(i|me|my|myself|we|us|our|ours|ourselves|you|your|yours|yourself|yourselves|he|him|his|himself|she|her|hers|herself|it|its|itself|they|them|their|theirs|themselves|what|which|who|whom|whose|this|that|these|those|am|is|are|was|were|be|been|being|have|has|had|having|do|does|did|doing|will|would|should|can|could|ought|i'm|you're|he's|she's|it's|we're|they're|i've|you've|we've|they've|i'd|you'd|he'd|she'd|we'd|they'd|i'll|you'll|he'll|she'll|we'll|they'll|isn't|aren't|wasn't|weren't|hasn't|haven't|hadn't|doesn't|don't|didn't|won't|wouldn't|shan't|shouldn't|can't|cannot|couldn't|mustn't|let's|that's|who's|what's|here's|there's|when's|where's|why's|how's|a|an|the|and|but|if|or|because|as|until|while|of|at|by|for|with|about|against|between|into|through|during|before|after|above|below|to|from|up|upon|down|in|out|on|off|over|under|again|further|then|once|here|there|when|where|why|how|all|any|both|each|few|more|most|other|some|such|no|nor|not|only|own|same|so|than|too|very|say|says|said|shall)"
},
{
"type": "formula", "as": "angle",
"expr": "[-45, 0, 45][~~(random() * 3)]"
},
{
"type": "formula", "as": "weight",
"expr": "if(datum.text=='VEGA', 600, 300)"
}
]
}
],
"scales": [
{
"name": "color",
"type": "ordinal",
"domain": {"data": "table", "field": "text"},
"range": ["#005aa0", "#8cb423", "#5a696e"]
}
],
"marks": [
{
"type": "text",
"from": {"data": "table"},
"encode": {
"enter": {
"text": {"field": "text"},
"align": {"value": "center"},
"baseline": {"value": "alphabetic"},
"fill": {"scale": "color", "field": "text"}
},
"update": {
"fillOpacity": {"value": 1}
},
"hover": {
"fillOpacity": {"value": 0.5}
}
},
"transform": [
{
"type": "wordcloud",
"size": [800, 400],
"text": {"field": "text"},
"rotate": {"field": "datum.angle"},
"font": "Helvetica Neue, Arial",
"fontSize": {"field": "datum.count"},
"fontWeight": {"field": "datum.weight"},
"fontSizeRange": [12, 56],
"padding": 2
}
]
}
]
}
```
Show rendered plot
Bar Chart
Show code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
```vega
{
"$schema": "https://vega.github.io/schema/vega/v5.json",
"description": "A nested bar chart example, with bars grouped by category.",
"width": 300,
"padding": 5,
"autosize": "pad",
"signals": [
{
"name": "rangeStep", "value": 20,
"bind": {"input": "range", "min": 5, "max": 50, "step": 1}
},
{
"name": "innerPadding", "value": 0.1,
"bind": {"input": "range", "min": 0, "max": 0.7, "step": 0.01}
},
{
"name": "outerPadding", "value": 0.2,
"bind": {"input": "range", "min": 0, "max": 0.4, "step": 0.01}
},
{
"name": "height",
"update": "trellisExtent[1]"
}
],
"data": [
{
"name": "tuples",
"values": [
{"a": 0, "b": "a", "c": 6.3},
{"a": 0, "b": "a", "c": 4.2},
{"a": 0, "b": "b", "c": 6.8},
{"a": 0, "b": "c", "c": 5.1},
{"a": 1, "b": "b", "c": 4.4},
{"a": 2, "b": "b", "c": 3.5},
{"a": 2, "b": "c", "c": 6.2}
],
"transform": [
{
"type": "aggregate",
"groupby": ["a", "b"],
"fields": ["c"],
"ops": ["average"],
"as": ["c"]
}
]
},
{
"name": "trellis",
"source": "tuples",
"transform": [
{
"type": "aggregate",
"groupby": ["a"]
},
{
"type": "formula", "as": "span",
"expr": "rangeStep * bandspace(datum.count, innerPadding, outerPadding)"
},
{
"type": "stack",
"field": "span"
},
{
"type": "extent",
"field": "y1",
"signal": "trellisExtent"
}
]
}
],
"scales": [
{
"name": "xscale",
"domain": {"data": "tuples", "field": "c"},
"nice": true,
"zero": true,
"round": true,
"range": "width"
},
{
"name": "color",
"type": "ordinal",
"range": "category",
"domain": {"data": "trellis", "field": "a"}
}
],
"axes": [
{ "orient": "bottom", "scale": "xscale", "domain": true }
],
"marks": [
{
"type": "group",
"from": {
"data": "trellis",
"facet": {
"name": "faceted_tuples",
"data": "tuples",
"groupby": "a"
}
},
"encode": {
"enter": {
"x": {"value": 0},
"width": {"signal": "width"}
},
"update": {
"y": {"field": "y0"},
"y2": {"field": "y1"}
}
},
"scales": [
{
"name": "yscale",
"type": "band",
"paddingInner": {"signal": "innerPadding"},
"paddingOuter": {"signal": "outerPadding"},
"round": true,
"domain": {"data": "faceted_tuples", "field": "b"},
"range": {"step": {"signal": "rangeStep"}}
}
],
"axes": [
{ "orient": "left", "scale": "yscale",
"ticks": false, "domain": false, "labelPadding": 4 }
],
"marks": [
{
"type": "rect",
"from": {"data": "faceted_tuples"},
"encode": {
"enter": {
"x": {"value": 0},
"x2": {"scale": "xscale", "field": "c"},
"fill": {"scale": "color", "field": "a"},
"strokeWidth": {"value": 2}
},
"update": {
"y": {"scale": "yscale", "field": "b"},
"height": {"scale": "yscale", "band": 1},
"stroke": {"value": null},
"zindex": {"value": 0}
},
"hover": {
"stroke": {"value": "firebrick"},
"zindex": {"value": 1}
}
}
}
]
}
]
}
```
Show rendered plot
Taken from https://vega.github.io/vega/examples/nested-bar-chart/.
C4 with PlantUML
Show code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
```plantuml
@startuml "techtribesjs"
!include https://raw.githubusercontent.com/plantuml-stdlib/C4-PlantUML/master/C4_Container.puml
' uncomment the following line and comment the first to use locally
' !include C4_Container.puml
LAYOUT_TOP_DOWN()
'LAYOUT_AS_SKETCH()
LAYOUT_WITH_LEGEND()
Person_Ext(anonymous_user, "Anonymous User")
Person(aggregated_user, "Aggregated User")
Person(administration_user, "Administration User")
System_Boundary(c1, "techtribes.js"){
Container(web_app, "Web Application", "Java, Spring MVC, Tomcat 7.x", "Allows users to view people, tribes, content, events, jobs, etc. from the local tech, digital and IT sector")
ContainerDb(rel_db, "Relational Database", "MySQL 5.5.x", "Stores people, tribes, tribe membership, talks, events, jobs, badges, GitHub repos, etc.")
Container(filesystem, "File System", "FAT32", "Stores search indexes")
ContainerDb(nosql, "NoSQL Data Store", "MongoDB 2.2.x", "Stores from RSS/Atom feeds (blog posts) and tweets")
Container(updater, "Updater", "Java 7 Console App", "Updates profiles, tweets, GitHub repos and content on a scheduled basis")
}
System_Ext(twitter, "Twitter")
System_Ext(github, "GitHub")
System_Ext(blogs, "Blogs")
Rel(anonymous_user, web_app, "Uses", "HTTPS")
Rel(aggregated_user, web_app, "Uses", "HTTPS")
Rel(administration_user, web_app, "Uses", "HTTPS")
Rel(web_app, rel_db, "Reads from and writes to", "SQL/JDBC, port 3306")
Rel(web_app, filesystem, "Reads from")
Rel(web_app, nosql, "Reads from", "MongoDB wire protocol, port 27017")
Rel_U(updater, rel_db, "Reads from and writes data to", "SQL/JDBC, port 3306")
Rel_U(updater, filesystem, "Writes to")
Rel_U(updater, nosql, "Reads from and writes to", "MongoDB wire protocol, port 27017")
Rel(updater, twitter, "Gets profile information and tweets from", "HTTPS")
Rel(updater, github, "Gets information about public code repositories from", "HTTPS")
Rel(updater, blogs, "Gets content using RSS and Atom feeds from", "HTTP")
Lay_R(rel_db, filesystem)
@enduml
```
Show rendered plot
Taken from https://github.com/plantuml-stdlib/C4-PlantUML.
GraphViz
Show code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
```graphviz
digraph G {
concentrate=True;
rankdir=TB;
node [shape=record];
140087530674552 [label="title: InputLayer\n|{input:|output:}|{{[(?, ?)]}|{[(? , ?)]}}"];
140087537895856 [label="body: InputLayer\n|{input:|output:}|{{[(?, ?)]}|{[(?, ?)]}}"];
140087531105640 [label="embedding_2: Embedding\n|{input:|output:}|{{(?, ?)}|{ (?, ?, 64)}}"];
140087530711024 [label="embedding_3: Embedding\n|{input:|output:}|{{(?, ?)}|{ (?, ?, 64)}}"];
140087537980360 [label="lstm_2: LSTM\n|{input:|output:}|{{(?, ?, 64)}|{(?, 128)}}"];
140087531256464 [label="lstm_3: LSTM\n|{input:|output:}|{{(?, ?, 64)}|{(?, 32) }}"];
140087531106200 [label="tags: InputLayer\n|{input:|output:}|{{[(?, 12)]}|{[(? , 12)]}}"];
140087530348048 [label="concatenate_1: Concatenate\n|{input:|output:}|{{[(?, 128), (?, 32), (?, 12)]}|{(?, 172)}}"];
140087530347992 [label="priority: Dense\n|{input:|output:}|{{(?, 172)}|{(?, 1) }}"];
140087530711304 [label="department: Dense\n|{input:|output:}|{{(?, 172)}|{(?, 4)}}"];
140087530674552 -> 140087531105640;
140087537895856 -> 140087530711024;
140087531105640 -> 140087537980360;
140087530711024 -> 140087531256464;
140087537980360 -> 140087530348048;
140087531256464 -> 140087530348048;
140087531106200 -> 140087530348048;
140087530348048 -> 140087530347992;
140087530348048 -> 140087530711304;
}
```
Show rendered plot
Taken from https://graphviz.org/Gallery/directed/neural-network.html.
BlockDiag
Show code
1
2
3
4
5
6
```blockdiag
blockdiag {
A -> B -> C -> D;
A -> E -> F -> G;
}
```
Show rendered plot
Taken from http://blockdiag.com/en/blockdiag/examples.html.
Deployment
The service is provided by HIFIS Software and is hosted entirely on HZDR servers. It does not share any data with external providers. As usual, the deployment can be reproduced via the corresponding project - only accessible for employees within Helmholtz.
Comments and Suggestions
If you have suggestions, questions, or queries, please don’t hesitate to write us.