When you need a proper set of Kubernetes icons, just pick them up at GitHub.
This Repo contains a full set of Kubernetes icons to be used under Apache-2.0 or CC-BY-4.0.
You will find png and svg files as well as an extensive Google slide deck with additional information,
For the last couple of years, I have seen many so-called “architecture” diagrams including those suffering from inconsistent notation, unclear depiction lack of information and ambiguity.
The worst scenario was in my last role, where I dealt with company-wide application integration. Each and every team had their own way to draw (or not to draw) security architectures. Guidelines? Nope. Nervous breakdowns n a daily base if something went wrong? Definitely.
So he/she who’s name is not known created a quite comprehensive PDF guideline which helps you to draw more precise security related architectural diagrams. Definitely worth a look.
Link: https://github.com/chaocipher/JunkDrawer/blob/master/Security%20Diagramming%20Guidance%20(2019).pdf
Simon Brown -inventor of the C4 model – recently came up with a checklist about diagrams and notations.
A few people have asked me for this recently so … if you’re looking for my software architecture diagram notation checklist, you can find it here…
Although it seems obvious what is checked in this list, almost every single diagram I have encountered in my career did not provide all the information. Copy it, print it, as an architect (and software developer) carry it with you – and use it.
Link: https://c4model.com/assets/software-architecture-diagram-review-checklist.pdf
After prototyping quite a lot, I finally came up with a simple architectural draft of the software for the first version of my home automation project.
As this is a 10,000 feet view, I do not bother about the technical details for right now. I thought of two main components TheWeb and TheHub.
TheHub would be the one application, collecting data all the time from an arbitrary number of local sensors and controlling various actuators. At the same time it would push data to TheWeb to make it accessible to the outer space. I have thought of pros and cons about a push, a pull or even a pub/sub mechanism. For now pushing data will be much easier. Also, as this won’t be my main interface for interacting with the environment, there is no need to be up to date by the second.
The second component, TheWeb, will be a REST-architectural style Web service I will push the data from TheHub to, and which will provide data to the actual Web application. Also, the service will chat with various external services out there in the wild.
I was recently asked which notation I am using to draw my sketches: It’s called WebComposotion Architectur Model or simply WAM. It’s quite a simple notation for outlining architectural designs and dependencies in distributed and federated systems which include organizational boundaries. There is a series of publications, introducing the concepts behind WAM (my my book deals quite a lot with WAM as well):
In a nutshell, it is supposed to be a very easy to draw language to communicate architectural decisions. Maybe that’s something for another post, though.
With this basic design, I am ready to come up with some bits soon.
Over the last few days I (re-)thought a lot of the role of an architect in a Scrum team. I tried to avoid to read about other opinions and just thought of my very own experience over the last few years in agile teams.
There should be an architect in your organization, however, he should not be part of the Scrum team itself. Scrum is about the development process in your organization or project and as such is it a required constraint for the architecture to develop.
The architect deals with non technical issues, so called facts of life, politics, organizational constraints, budget restrictions and so on. No team member could address thees in the regular development sprints.
As an architect one should analysis and manage risks – again nothing a team member could do during a regular two or four week sprint. Considering risks probably begins long before the team is Assembled and t he project is started. In fact, as an architect one could even point out the project might not doable due to various reasons.
An architect has to run in iterative cycles as constraints change, customers come up with new and modified requirements and organizational goals mit shift over time. All this might require some redesign or evolution of the architecture. However, these architectural cycles are not bound to the development cycles of e team. They are more related to the business, customers and organization.
The architect is a technical leader. Maybe by prototypes or bullet tracing he shows how hard parts of the system can or will be addressed. As such he provides a base to the team to better estimate the complexity of the upcoming implementation. Eventually, as architect it is important to address the hardest issues first whiled the Scrum team addresses the tasks with the highest business value first.
As an architect you teach and coach your team. In Martin Fowler's article Who Needs an Architect?, published in IEEE Software, Fowler points out, that an good architect mentors his team, not sitting in his ebony tower.
Dealing with uncertainty is probably one of the most underestimated aspects of the dayjob of an architect. In the role of an architect, I have to make decisions under a high degree of uncertainty. As many aspects of a project change over time, early decisions are uncertain. However, as an architect, you have to consider the consequences of these decisions. It's a major part of your job to consider risks and work on plans wether any event occures that affects your project both, a a threat or a chance.
I contrast, in the role of an developer, you should not be confronted with uncertainty. You have a tough schedule and proper tasks to fulfill, Technologie and tools are in place and in the best case you have all the knowledge to perfomreyou task. If the requirements for your task are not clear, you probably cannot fulfill it.
I have experienced this very issue during the last few years several times. Most of the time this was caused by the lack of an architect or the position of the architect was not dealing with the role of the architect in a proper way, most of the time writing code themselfs.
Therefore, do architects write code? This might be one of the most discussed topics for software engineering. Personally, I have experienced both, architects writing (productive) code all the time a well as architects never worked in product development. As an architect you probably should provide certain coding skills. Earning your street cred before becoming an architect is inevitable. You must be able to read, understand and improve the code. However, you have to delegate the actual task of building the code base to others. Nevertheless, you probably have to improve your coding skills permanently. Therefore, an erchitect must be able to write excellent code, however he should not write the bits delivers despite prototypes and tracer bullets.
As a fact, running an agile environment does not supersede an architect. Neither does it reduntize the overall product planing and designing process. The architect is not part of the Scrum team as he does not deliver within the Scrum team. He is in a position existing before the Scrum team is assembled, he leads the direction and can hold an consulting position for the team. However, the a architect is not part of the sprint plan and as he is not part of the Scrum team as such.
Looking at myself, I see how different I work nowadays with devices than almost 30 years ago. In the early days of personal computers you spend a lot of time in figuring out what you actually can do with your Commodore C64 or your very first 286 hardware while knowing each component's specification. Nowadays it is simply about the available software. Most of the users probably do even not know about any technical details of the device they are using beside if it is slow or fast.
If you look at professionals who use computers, they often use one specific application, which maybe is shut down only once at the end of the week. Personal users probably don't know that there are more applications on the computer than the web browser.
As computer professionals we tend to forget to think about the why others do use computers. We see the full potential of the latest programming language, the computing power, the maximum available bandwidth and all the fancy features we know about.
Tablets such as the iPad or the new Nexus are great for end users. Quite intuitive to use, and no need to worry about the hardware. Whatever users want to to, they simply have to find the right app. I fact, I use my iPad for many common tasks, even for writing, blogging and editing images the apps are meanwhile quite well done.
Specialized applications used by various professionals do not need a fully equipped personal computer. Ever looked at a doctor's place? In every surgery you might find a personal computer running often just one program. Or have a look at a common electronic it furniture megastore. Each information desk will probably has one personal computer running one program on it. Often, these programs are typically host applications where the client continually requests information from a server application
There is no reason to put a fully equipped computer in every room for a single application. Either a thin client or some lightweight tablet might the answer here. Either a web hosted application or a small application communicating with a server (e.g. In the cloud) might be a good solution.
As professional software architects and designers we should consider this while designing application even if stakeholders still request old fashioned desktop applications.
Recently, I had a interesting conversation about interface inheritance with one of my colleagues. Reason was a decision necessary how to implement basic behavior based on a set of interfaces for a number of classes. At first, I was not comfortable to let one interface inherit from another. I was quite biased from the design of the code base I am currently working on.
Generally spoken, each class inherits from its very own interface. In addition the inheritance scheme of classes is reflected in the interface inheritance as seen in the example below. Why would one come up with such a design at all?
To understand this design (it’s still a design) some more context is required. In the particular codebase a dependency injection container is used to resolve instances of a particular type. To do so a unique identifier is required. This could be a string but also an interface. E.g. the Microsoft Extensibility Framework (MEF) makes usage of interfaces for resolving. Using MEF it is quite easy to get a set of components implementing a particular interface (e.g. some kind of IPlugin interface).
The issue I’ve seen, there are only few common interfaces in the codebase . Instead of collecting all types of a particular interface, dedicated interfaces are used to resolve particular instances of types. However, by using interfaces this way, the focus is to identify types by their interfaces not using interfaces as contracts.
Back to the issue how to design the interface inheritance, we came up with two alternate approaches, both valid indeed, but with very different design goals.
In the left hand approach two types inherit from the same base type implementing a particular interface. Both types also implement another interface, i.e. both types fulfill the same contract. In the right hand approach we see both types inheriting from the same interfaces as well. While at the end both approaches will end in the same a similar result, there is a significant difference in the semantic.
Based on both approaches, we came up with two possible solutions for a .NET implementation. While this might seem quite academic to you, there is quite a difference how one might use these types.
As in the example before, both approaches will end up in two classes implemented identically, however, both implementations show semantic differences best seen when considering the usage of these classes. In the left hand example one could iterate through a typed list of IAlgorithm calling a Dispose method required by the IDisposable interface. This implementation is obviously contravariant. Following the right hand scheme, you might still iterate through the list, however, before accessing the Dispose method it is required to cast the concrete instance to IDisposable. While being still contravariant, it is not implicitly possible.
The question now, is to decide when to use the fist or the second approach. Interfaces inheriting from other interfaces is absolutely valid once you can answer the question if your type is some kind of with yes. In the given example each algorithm is an IDisposable. No exception, no excuse. Choosing the second approach you should be able to answer the question whether your type needs to fulfill a particular contract with yes. If only a few algorithms need to be fulfill the contract given by the IDisposable interface, and an algorithm is not a IDisposable by default, the second approach might be the right to choose. While each algorithm is still an IAlgorithm, only some of them could implement the IDisposable interface.
Maybe this seems obvious to you, however, I still see quite experienced developers having significant problems in choosing appropriate inheritance structures. From avoiding inheritance at all to the point of using the most complex inheritance structures you might have ever seen in your programmers life. There is no right or wrong but there might be a best solution suitable to your problem. So never hesitate questioning your current design looking for a better approach.
Ok, how would you describe the core architecture of your current project? Inspired by Ralf’s idea, I really thought about the WebComposition/DGS architecture. What are the e core concepts and the key ideas of it? If everything has to fit on a napkin, you really have to restrict yourself in drawing different components. The interactive NapkinNotebook thus cam in very handy. Let’s look at the first try of the architecture sketch?
What are the core components we find in the WebComposition/DGS? Well, there is the Data Grid Service (DGS) as central entity? What else? Some storage solution for data and some other storage solution for metadata. Ok, here we have to think the first time? Two of them? Definitely yes. Metadata might be saved a completely different way than the data itself. You don’t think so? Let’s see, think about our images as digital photos and the metadata created from the EXIF data extracted from the imaged. The EXIF data is stored as RDF in some triple store, the images are stored on the file system. Consequently, we need two storage solutions.
OK, next component on the diagram: Users; do the interact directly with the DGS? No, usually the use a Web-based application. The user also might be another application or a service – but that’s no information to be on this sketch. The important thing is: there is usually only interaction through some other component – usually a Web-based application.
After some feedback from Ralf, I’ve updated my sketch using another software cell for the Web application. Makes much more sense though, what do you think?
I am going to implementing a publish/subscribe mechanisms for our recent Web-based research prototype. Therefore, I am interested what others already thought about this pattern. Looking for the pub/sub paradigm, Google returns the corresponding Wikipedia article on the first place. Screening the article, there are a few starting points worth to be remembered:
Looking for more information on the Web I cam across the Publish/Subscribe integration pattern from Microsoft’s patterns & practices. The problem statement seems reasonable:
In the context description, the following communication infrastructures are mentioned:
In contrast to to the Wikipedia article, we learn about three different types of mechanisms:
Having a closer look, the List-based Publish/Subscribe mechanism maintains a list of subscribers, similar to the Observer pattern. Attach() and Detach() operations allow to modify the list of subscribers while a Notify() operation is used to send updates to the subscribers. Seems to be well suited if you have one publisher and many subscribers, but does not look suitable if subscribers watch many subjects. The core functionality of List-based Publish/Subscribe can thus be identified as
If we understand subscription lists as named channels, the List-based Publish/Subscribe represents a topic-based subscription mechanism.
The Broadcast-bases Publish/Subscribe mechanism simply dumps messages to the local are network. Each subscriber is responsible for listening and inspecting the subject line of the message. If the subject line matches, the subscriber processes the message. This approach seems to be a optimum in decoupling the system. Clearly, this can be identified as some kind of a topic-based system. If the publisher needs to know about subscribers to a particular topic, a hybrid approach can be chosen, where a additional process requests information about interested subscribers. To establish the hybrid system, however, every subscriber must respond to the request. Another name mentioned in the article is publish/subscribe channel with reactive filtering due to the responsibility of each subscriber to filter the messages on its own.
In difference to the Wikipedia article, in this article the author differentiates between topic-based and content-based mechanisms. In this context, both, List-based Publish/Subscribe and Broadcast-based Publish/Subscribe are understood as topic-based mechanisms.
While topics are considered as a pre-defined set of subjects, each message in a content-based system can be understood as a single dynamic logical channel. This idea was proposed in The Evolution of Publish/Subscribe Communication Systems [pdf]. We will come back to this paper later.
Where to implement the pub/sub functionality depends on your underlying communication structure:
The article also differentiates between fixed subscriptions and dynamic subscriptions. While applications cannot control their subscriptions, dynamic subscriptions allow to modify subscriptions through certain control messages.
Some more keywords are listed in the article:
How to implement a dynamic list-based publish-subscribe pattern is illustrated in the MSDN library.
I found also some article about the way EDA (event-driven architecture) extends SOA including a nice depiction of the idea behind EDA. There, EDA is proposed for a publish/subscribe mechanism rather than a command/control mechanism as provided by SOA. EDA seems especially suitable when you are facing
It is also mentions that some good support for the EDA pub/sub pattern would be a declarative model.
I also came along this article giving a brief overview of Publish-Subscribe Channel pattern from the book Enterprise Integration Patterns by G. Hohpe and B. Woolf. It basically tells that the channel delivers a copy of the message to each of the output channels where each output channels has only one subscriber. After the message is consumed, it is removed from the channel.
Having a look into Exploiting Virtual Synchrony in Distributed Systems, mentioned in the beginning, gives you an insight into several issues in distributed systems. One interesting fact to bear in mind is about synchrony vs. asynchrony. If your publisher requires responses this could be 0, 1 or n for n subscribers. If you expect 0 responses you actually run a asynchronous system. For so-called process groups an interface is provided, allowing to join or leave a group but also to receive updates on the group memberships. Sounds similar? The Observer pattern, I see here. In the described news service, one already realizes the common concepts described before: “Each subscriber receives a copy of any message having a ‘subject’ for which it has enrolled on the order they were posted.“. The overall description is rather abstract, but gives a interesting insight into the development of the mechanism.
Afterwards I ended up directly with The Evolution of Publish/Subscribe Communication Systems, providing a well written summary of the publish/subscribe paradigm. Especially the decoupling fact has been structured into
Again, we see content-based and topic-based mechanisms which makes me think twice of the classification proposed in the Wikipedia article. Back to the paper, the authors state that content-based pub/sub systems cannot rely on
A single server simply cannot deal with a high number of subscribers and the limited number of IP multicast addresses does not fit the large number of logical channels. Rather they propose a application-level realization through a set of event-brokers, exchanging information on a point-to-point basis. For broker interaction the following issues are pointed out:
Maybe its worth to mention, that both papers address communication systems on network and overlay network infrastructure-levels than on application-level. However, the concepts are the same.
Baldoni comes up with the concept of ad-hoc subscription languages, compared to SQL for databases. This, however, requires a-priori knowledge of the structure of the information space. At least, the idea of selecting subscriptions or topics using a query language sounds quite appealing. As future research direction, a potential formal specification of the subscription service, provided by a pub/sub system is proposed.
Another often cited paper I have a look at is The Many Faces of Publish/Subscribe [pdf]. Similar to the paper before, the three decoupling dimensions time, space and synchronization are considered to extract the common concepts of different variants of the pub/sub paradigm. We learn that individual point-to-point and synchronous communication leads to rigid and static applications. Three types of pub/sub mechanisms are introduced:
The basic terms for sending and receiving messages through a software bus/event used here are
The core system should provide a
The events used here are called subscribe(), unsubscribe() and publish() – not that different from the ones we know from the Observer pattern. Some new operation is called advertise() to advertise the nature of future events of an publisher. That way, the event service can adjust to the expected event flows and subscribers can learn when new types of information come available. We also learn about alternative communication paradigms here:
For the three pub/sub forms we fi
nd some more detailed information:
Having a closer look at events we learn about the classification into messages (delivery through a single operation e.g. notify) and invocations (event triggers some specific operation on the subscriber). Furthermore, invocations are directed to a certain kind of objects and provide some well-known semantics. You can also differentiate between on-way invocations (COM+ or CORBA Event Service) and those requiring some return value.
We see different kinds of architectures there:
Dissemination of messages is also discussed but relies a lot on the underlying concepts. Efficient multicast in content-based pub/sub systems, however, is pointed out to be still an issue.
Some more points to be considered are related to QoS:
Bearing this information in mind, I now have a look into the Publish-Subscribe Notification for Web Services [pdf] whitepaper as part of the WS-Notification family. The document deals with the notification pattern for notification-based or event-driven systems in the Web service context. Here we see the same pattern as learned before:
The spec defines (among others) some interesting requirements:
In the terminology section we find another interesting statement saying “a Subscription is a WS-Resource” where a WS-Resource is defined as follows:
“A Web service having an association with a stateful resource, where the stateful resource is defined by a resource properties document type and the association is expressed by annotating a WSDL portType with the type definition of the resource properties document”
Got it? At least let us think of subscriptions as resources. This idea lines up well with my current research.
Also the fact of hierarchically structured topics is considered: Especially topic trees are hierarchically structured topics and topic spaces are a set of topic trees grouped together into the same namespace (obviously due to administrative reasons).
It is actually the first document dealing with security aspect, listing the following classes of attacks:
Finally, I found the Distributed Publish/Subscribe Event System on CodePlex: In the whitepaper, the various types of pub/sub are characterized by
The Web Solutions Platform (WSP) is designed as a distributed pub/sub system and works both, intra-machine and inter-machine. Applications here subscribe to event types, so it looks like an event-based pub/sub system. The document provides some more descriptions on the system itself but no more information on publish/subscribe mechanism in general.
That’s a lot of stuff and now I have to spend some time in reflecting all these information for my design.