retgits.com

Streamline Unified Data Governance with AWS Lake Formation and Dremio

Thu, 01 May 2025 00:00:00 +0000

Customers are building large data lakes on Amazon Web Services (AWS) to democratize their access to data. As a result of that, data governance becomes increasingly important. Customers need to know data is accessed at the right time, by the right people, and in the right context. To implement fine-grained data access permissions, customers use AWS Lake Formation. AWS Lake Formation provides data access controls for AWS services like Amazon Redshift, Amazon Athena, and Amazon EMR. It also offers data access controls for AWS Partners like Dremio.

AWS Lake Formation 2023 Year in Review

Thu, 18 Jan 2024 00:00:00 +0000

AWS Lake Formation and the AWS Glue Data Catalog form an integral part of a data governance solution for data lakes built on Amazon Simple Storage Service (Amazon S3) with multiple AWS analytics services integrating with them. In 2022, we talked about the enhancements we had done to these services. We continue to listen to customer stories and work backwards to incorporate their thoughts in our products. In this post, we are happy to summarize the results of our hard work in 2023 to improve and simplify data governance for customers.

What's new in AWS Lake Formation (reInvent 2023)

Wed, 29 Nov 2023 00:00:00 +0000

Chief data officers, data platform administrators, architects, owners, and consumers are looking to simplify data access permissions and governance. AWS Lake Formation makes it easier to centrally govern, secure, and globally share data for analytics and machine learning use cases. Join this session to learn about new capabilities, customer stories, and how you can get the most out of Lake Formation.

Governing Databricks Data Access with AWS Lake Formation and Privacera (AWS Partner Network Blog)

Fri, 04 Aug 2023 00:00:00 +0000

Many organizations have standardized or plan to standardize their unified data security governance on AWS Lake Formation, which provides powerful data access control to Amazon Redshift, Amazon Athena, and Amazon EMR.

Some of these organizations are also leveraging Databricks, however, and would like to create and manage data access policies for Databricks using AWS Lake Formation as well. They want to have consistent policy enforcement and monitoring across their AWS services, Databricks, and Amazon Simple Storage Service (Amazon S3).

In this post, we will discuss the AWS Lake Formation and Privacera integrated solution that extends AWS Lake Formation source support to Databricks. It provides data access policy authorship and maintenance from one safe and convenient location, AWS Lake Formation.

Protecting Sensitive Data with Collibra Protect and AWS Lake Formation (AWS Partner Network Blog)

Fri, 16 Jun 2023 00:00:00 +0000

What’s the point of data if you can’t get your hands on (or mind around) it?

In today’s data-driven world, ensuring the security and proper management of sensitive information is paramount. Collibra Protect and AWS Lake Formation offer a powerful combination to address the growing challenges of enterprise data access governance.

Collibra Protect, part of the Collibra Data Intelligence Cloud, protects sensitive data and makes it available, or partially available, to specified groups of users. AWS Lake Formation is a fully managed serverless service that allows you to build clean and secure data lakes in days.

Privacera Launches 2 New Solutions for AWS Lake Formation

Fri, 02 Jun 2023 00:00:00 +0000

Many organizations are using AWS Lake Formation to manage data security and access management for Amazon Athena, Amazon Redshift Spectrum, or Presto with Amazon EMR, but they want to be able to manage other sources with a single central data security platform. Thus, allowing organizations to apply consistent and un-siloed data security and access policies across all their data sources, reduce the effort required to manage data security and access policies, make data more accessible, and enhance their security posture.

Achieving your modern data architecture

Thu, 18 Aug 2022 00:00:00 +0000

A modern data strategy is a comprehensive plan or blueprint to help you manage, access, analyze, and act on data. Many companies are developing roadmaps to help them realize their modern data goals. In this session, learn how deploying a modern data architecture on AWS can help businesses navigate their data challenges, optimize analytics processes, and deliver business insights faster. Take a closer look at AWS Glue and AWS Lake Formation, and see how they help accelerate the process.

How To Use Innovation And Proven Methodologies To Uncover Your Distinctive Competencies

Thu, 28 Jul 2022 00:00:00 +0000

One of the largest struggles within product is articulating your organization’s unique abilities to deliver value to its market. But guess what, it’s also one of the most important! So how do you develop a successful path that combines innovation, proven methodology and sure-fire approaches to hone-in on the attributes and differentiators that set you apart from your competitors?

Simply Stateful Serverless

Thu, 28 Oct 2021 00:00:00 +0000

One of the things I love about serverless is that I never have to be bothered with managing servers, it’s just using a service like Lambda, Cloud Run, etc and my code is running. If I want to use a database I can rely on services like DynamoDB or CosmosDB. While I think that is absolutely great, it feels like serverless is only for stateless processes. I think serverless needs a bold and stateful vision so that we can build any type of application (stateful and stateless) without ever managing servers. In this talk, I’ll touch on why statefulness matters and how stateful serverless makes patterns like Event Sourcing and CQRS available to anyone.

Why (stateful) serverless matters for server admins

Tue, 21 Sep 2021 00:00:00 +0000

In the session I went over why serverless is important to our industry, why server admins (which I then rephrased to SREs) are so important to our serverless success, and why stateless isn’t the answer for everything. Technology wise I’ll be “all over the map” talking about things like Knative and the VMware Event Broker Appliance, AWS Lambda, Akka Serverless

Lightbend’s Akka Serverless enables stateful app development without a database - SiliconANGLE

Thu, 10 Jun 2021 00:00:00 +0000

Leon Stigter, senior product manager for serverless at Lightbend, elaborated on the problem, telling SiliconANGLE that developers generally consider serverless to be a “stateless solution,” which means that every time an application needs to do something, it must connect to a database first. For single services that isn’t so difficult, he said, but when it happens at bigger scales, tasks such as connection pooling can be very difficult, causing lots of headaches.

How To Secure Akka Serverless Apps With Auth0

Tue, 08 Jun 2021 00:00:00 +0000

As Auth0 says on their website “Identity is the front door of every user interaction”. As you’re building our new serverless applications, that becomes even more important as you have multiple apps that you need to secure. In this blog post I’ll walk you through how that can work in Akka Serverless.

How To Set Up Continuous Integration and Delivery With Github Actions and Akka Serverless

Tue, 01 Jun 2021 00:00:00 +0000

CI/CD is one of those quintessential mindset shifts that helps developers automate away the toil of deploying apps. Especially in the realm of serverless, where the whole idea is to focus on the things that matter and let the undifferentiated heavy lifting be handled by others, automating as much as possible is paramount. It helps developers focus on what matters, code, and it helps business focus on what matters, getting quality software to market faster. So how does that work in Akka Serverless?

Thinking Stateful Serverless @ Micro.Sphere.IT

Wed, 09 Dec 2020 00:00:00 +0000

As developers, we all want to be more productive. Serverless helps you do just that, by letting you focus on the business logic while shifting operations somewhere else. As more companies discover this emerging technology, we also discover drawbacks like state management. In this session, I focused on what serverless is, how it helps developers, what potential drawbacks exist, and how we can add state management into serverless.

Test-driving Event-Driven Apps on Kubernetes

Thu, 15 Oct 2020 00:00:00 +0000

As developers, we all want to be more productive. Knative, a Kubernetes based platform to deploy and manage modern serverless works, helps to do just that. The idea behind Knative is to abstract away the complexity of building apps on top of Kubernetes as much as possible and Tekton is a powerful and flexible open-source CI/CD tool. How can you bring those two together on your local machine to try a few things out or even develop your apps? During this talk, we looked at setting up a KinD cluster, bootstrapping Knative and Tekton, and deploying an app!

Deploying your first app on the Kubernetes based Knative platform

Thu, 13 Aug 2020 00:00:00 +0000

As developers, we all want to be more productive. Knative, a Kubernetes based platform to deploy and manage modern serverless works, helps to do just that. The idea behind Knative is to abstract away the complexity of building apps on top of Kubernetes as much as possible. How can you get Knative running on your local machine to try a few things out or even develop your apps? In this session, we ’ll look at setting up a Kubernetes cluster, installing Knative, and building an app.

Data Driven Decisions in DevOps @ MyDevSecOps

Thu, 02 Jul 2020 00:00:00 +0000

With everything going on in DevOps, I think we can safely say that building pipelines is the way to deploy your applications to production. But knowing what you deploy to production and whether it is actually okay needs more data, like security checks, performance checks, and budget checks. We’ve come up with a process for that, which we call Continuous Verification “A process of querying external systems and using information from the response to make decisions to improve the development and deployment process.” In this session, we’ll look at extending an existing CI/CD pipeline with checks for security, performance, and cost to make a decision on whether we want to deploy our app or not.

Automated DevOps for the Serverless Fitness Shop - Knowing what and why you go to production @ NS1 INS1GHTS 2020

Thu, 25 Jun 2020 00:00:00 +0000

In a nutshell, Continuous Verification comes down to making sure that DevOps teams put as many checks as possible into their CI/CD pipelines. These checks use external systems to validate the performance, security, and cost of your app without asking your engineers to do that manually. The systems that provide the data which decided whether your deployment goes to production or not, can also be used to help your engineers understand where the bottlenecks are in the process. With more checks in your automated pipeline, you have fewer manual tasks, less overhead, and better decisions to deploy to production or not. All that together means you get to spend more time at the beach!

Every Talk Has To Be Unique, Right?

Wed, 10 Jun 2020 00:00:00 +0000

Whether you’re a Product Manager or Developer Advocate, once you start you think that every presentation has to be unique… spoiler alert, it doesn’t have to be

How to set up a local Knative environment with KinD and without DNS headaches

Wed, 03 Jun 2020 00:00:00 +0000

Knative builds on Kubernetes to abstract away complexity for developers, and enables them to focus on delivering value to their business. The complex (and sometimes boring) parts of building apps to run on Kubernetes are managed by Knative. In this post, we will focus on setting up a lightweight environment to help you to develop modern apps faster using Knative.

Continuous Verification In A Serverless World @ Open Source Community Day

Thu, 30 Apr 2020 00:00:00 +0000

At VMware we define Continuous Verification as:

“A process of querying external systems and using information from the response to make decisions to improve the development and deployment process.”

At #OSSDay, I got a chance to not only talk about what that means for serverless apps and how you can build it into your existing pipelines using tools like GitLab, CloudHealth, Wavefront and Gotling.

Building a Serverless Fitness Shop - Observability

Wed, 15 Apr 2020 00:00:00 +0000

If you’ve read the blog posts on CloudJourney.io before, you’ve likely read the term “Continuous Verification”. If you haven’t that’s okay too. There’s an amazing article from Dan Illson and Bill Shetti on The New Stack explaining in detail what Continuous Verification is. In a nutshell, the Continuous Verification comes down to making sure that DevOps teams put as many checks as possible into their CI/CD pipelines. Adding checks into a pipeline means there are fewer manual tasks and that means you have access to more data tot smooth out and improve your development and deployment process.

So far we covered the tools and technologies, Continuous Integration, and Infrastructure as Code aspects of the ACME Serverless Fitness Shop. Now, it’s time to dive into observability!

Building a Serverless Fitness Shop - Infrastructure as Code

Wed, 08 Apr 2020 00:00:00 +0000

In part one we covered the tools and technologies and in part two we covered the Continuous Integration aspect of the ACME Serverless Fitness Shop. Now, it’s time to dive into Infrastructure as Code!

Building a Serverless Fitness Shop - Continuous Anything

Wed, 01 Apr 2020 00:00:00 +0000

Building a Serverless Fitness Shop - Tools and Tech

Mon, 23 Mar 2020 00:00:00 +0000

If you’ve read the blog posts on CloudJourney.io before, you’ve likely read the term “Continuous Verification”. If you haven’t that’s okay too. There’s an amazing article from Dan Illson and Bill Shetti on The New Stack explaining in detail what Continuous Verification is. To make sure we’re all on the same page, though, I’ll quickly go over it as well. As a definition, Continuous Verification is “A process of querying external system(s) and using information from the response to make decision(s) to improve the development and deployment process.”.

Tracking Distributed Errors In Serverless Apps

Mon, 09 Mar 2020 00:00:00 +0000

Microservices give us as developers an incredible amount of freedom. We can choose our language and we can decide where and when to deploy our service. One of the biggest challenges with microservices, though, is figuring out how things go wrong. With microservices, we can build large, distributed applications, but that also means finding what goes wrong is challenging. It’s even harder to trace errors when you use a platform like AWS Lambda.

Continuous Verification In A Serverless World @ Serverless Nashville

Thu, 27 Feb 2020 00:00:00 +0000

At VMware we define Continuous Verification as:

“A process of querying external systems and using information from the response to make decisions to improve the development and deployment process.”

At Serverless Nashville, I got a chance to not only talk about what that means for serverless apps but also how we use serverless in some of the business units at VMware.

Hybrid Security - From On-Prem to Serverless

Mon, 24 Feb 2020 00:00:00 +0000

DevOps, as a practice to build and deliver software, has been around for over a decade. What about adding security to that, though? After all, security is one of the cornerstones of today’s information technology. As it turns out, one of the first mentions of adding security was a Gartner blog post in 2012. Neil MacDonald wrote,

“DevOps must evolve to a new vision of DevOpsSec that balances the need for speed and agility of enterprise IT capabilities (…)”.

How To Build Infrastructure as Code With Pulumi And Golang - Part 2

Thu, 06 Feb 2020 00:00:00 +0000

Going into the series on creating Infrastructure as Code on AWS using Pulumi, I knew the team there was actively working on improving and expanding the Go support in Pulumi. What I didn’t realize is that it would be so quick and would be such a great improvement to the underlying code I needed to write. In this post, I’ll go over some of the code from my previous blog posts and update them to match the new SDK.

SAP Customer Experience Labs Talk – Episode 7 No Code / Low Code

Thu, 06 Feb 2020 00:00:00 +0000

One of my strong beliefs is that coding should be available to everyone. Whether that is a seasoned developer or someone who just wants to connect two systems together. With Project Flogo, we’ve made it possible for everyone to use the same constructs. If you want to use the web-based flow designer, that’s awesome! If you want to write your apps using the Go API, that’s awesome too. In this podcast I joined Jan Oberhauser (N8N), Nick O’Leary (Node Red), and the SAP Customer Experience Labs team to discuss No Code / Low Code.

How To Create AWS Lambda Functions Using Pulumi And Golang

Tue, 28 Jan 2020 00:00:00 +0000

I’ve looked at Pulumi to do a bunch of things, including creating subnets in a VPC, building EKS clusters, and DynamoDB tables. The one thing I hadn’t explored yet was how to deploy AWS Lambda functions using Pulumi, so that’s exactly what this blog is about.

Trusting your ingredients - What's in your function anyway?

Fri, 24 Jan 2020 00:00:00 +0000

As a developer, I’ve built apps and wrote code. As a cheesecake connoisseur, I’ve tried many different kinds of cheesecake. After I got to talk to some of the bakers, I realized that building apps and baking cheesecake have a lot in common. It all starts with knowing and trusting your ingredients.

According to Tidelift, over 90 percent of applications contain some open source packages. Developers choose open source because they believe it’s better, more flexible, and more extendible. A lot of developers also fear how well packages are maintained and how security vulnerabilities are identified and solved.

How To Create a DynamoDB Table In AWS Using Pulumi And Golang

Tue, 21 Jan 2020 00:00:00 +0000

In previous posts, I looked at Pulumi to do all sorts of things with infrastructure. Most apps, though, will need some form of datastore so in this post I’ll go over the steps to create a DynamoDB table in AWS using Pulumi.

How To Make Your AWS EKS Cluster Use Fargate Using Pulumi And Golang

Thu, 16 Jan 2020 00:00:00 +0000

At re:Invent, AWS introduced the ability to have EKS run pods on AWS Fargate, and Fargate is cheaper than hosting Kubernetes yourself. In the last post I created an EKS cluster, so let’s add this new capability to the cluster and remove the need to manage or provision infrastructure for our pods.

How To Create An AWS EKS Cluster Using Pulumi And Golang

Tue, 14 Jan 2020 00:00:00 +0000

Building a Kubernetes cluster is hard, so why not use a managed Kubernetes service? In the previous post I added subnets to a VPC. In this post, I’ll use the VPC to create an AWS EKS cluster.

How To Add Subnets To a VPC In AWS Using Pulumi And Golang

Thu, 09 Jan 2020 00:00:00 +0000

In the previous post, I looked at Pulumi to create a VPC. In this post, I’ll go over the steps to add some subnets to it.

How To Create a VPC In AWS Using Pulumi And Golang

Tue, 07 Jan 2020 00:00:00 +0000

The source code you write as a developer is important, but it is only one part of the entire application that goes into production. To deploy an app, you’ll need resources like API gateways, S3 buckets, or VPCs as well. Configuring those resources is a task you don’t want to do manually. How about building your infrastructure as code using the same language you’ve built your app in. That is what Pulumi allows you to do!

Cost Matters! The Serverless Edition

Mon, 09 Dec 2019 00:00:00 +0000

As a trend cloud vendors tend to use the word serverless quite loosely. While serverless comes in a lot of shapes and sizes and as long as the characteristics fit within the four categories from my last blog, it is a serverless service. To make sure that we’re all on the same page, I’ll use the following definition for serverless:

“Serverless is a development model where developers focus on a single unit of work and can deploy to a platform that automatically scales, without developer intervention.”

In this blog post, we’ll look at how that model works on AWS Fargate, which allows you to run containers without having to manage servers or clusters.

Serverless - From Microservice to Functions

Mon, 09 Dec 2019 00:00:00 +0000

Using serverless requires us to change our mindset on how we build apps and requires us to unlearn things we learned building apps in the past. At AWS re:Invent I got a chance to do a VMware Code session and talk about how we took part of our ACME Fitness Shop and transformed it into serverless functions with AWS Lambda.

Serverless - The Wrath of Containers

Mon, 09 Dec 2019 00:00:00 +0000

Containers were this awesome technology that ushered in the Cloud era and with a lot of new FaaS tools coming along, companies are wondering if they should jump the container ship altogether. As it turns out, containers definitely have value in Serverless. In this session we will take an existing containerized app and move it into AWS Fargate, look at the cost of running it, and how we can monitor it.

Why Serverless Architectures Matter

Mon, 04 Nov 2019 00:00:00 +0000

There are many predictions from market analyst firms on the size of the global serverless architecture market and how fast it will grow. The numbers range from $18B to $21.99B in the next few years with the compound annual growth rate (CAGR) in the double digits. But is serverless only a fancy name for products like AWS Lambda and Azure Functions?

Event-Driven Architectures - Putting Jazz Into Apps

Fri, 11 Oct 2019 00:00:00 +0000

The CTO of a company I have worked for used to say that services should be loosely coupled but tightly integrated. I didn’t realize until a lot later how true that statement is as you’re building out microservices. How those microservices communicate with each other has also changed quite a bit. More often than not, they send messages using asynchronous protocols. As an industry, we decided that this new way of building apps should be called “Event-Driven Architecture (EDA).”

Trusting Your Ingredients @DevOpsDays Columbus

Mon, 23 Sep 2019 00:00:00 +0000

As a developer, I always thought that security, like documentation, would be done by someone else. While that might have been true in the past, in today’s world that model no longer works. As a developer you’re responsible for the security of your app. Security in this case should be seen in the broadest sense of the word, ranging from licenses to software packages. A chef creating cheesecake has similar challenges. The ingredients of a cheesecake are similar to the software packages a developer uses. The preparation is similar to the DevOps pipeline, and recipe is similar to the licenses for developers. Messing up any of those means you have a messy kitchen, or a data breach!

How to send CloudFormation events to Wavefront using AWS Lambda

Fri, 20 Sep 2019 00:00:00 +0000

Imagine this, it’s 5pm on a Friday afternoon and while you really want to go enjoy the weekend, you also need to deploy a new version of your app to production. Using AWS CloudFormation (CF), you add a new instance to your fleet of EC2 instances to run your app.

GopherCon 2019 - Trusting Your Ingredients

Sat, 27 Jul 2019 00:00:00 +0000

Trusting Your Ingredients - What Building Go Apps And Cheesecake Have In Common.

In this lightning session at GopherCon 2019, I got the chance to talk about two things I love. Cheesecake and Golang! As a developer, I’ve written code and built apps, and I realized that building apps and creating a cheesecake have a lot in common. In both cases you need to have the right ingredients, you need to trust your suppliers and have transparency in your production process. In this talk, we’ll look at how you can, and why you should, know what is in the app you deploy.

Docker Meetup Group Atlanta - Trusting Your Ingredients

Tue, 09 Jul 2019 00:00:00 +0000

As a developer, I’ve written code and built apps, and I realized that building apps and creating a cheesecake have a lot in common. In both cases you need to have the right ingredients, you need to trust your suppliers and have transparency in your production process. I got to go to Atlanta and meet with the Docker Meetup Group there, where we got to talk about In this talk, how you can, and why you should, know what is in the app you deploy.

How to Get Webhooks Into Your System Using Solace PubSub+ Cloud

Wed, 05 Jun 2019 00:00:00 +0000

In some cases you might want to get some data from cloud based systems to an environment that doesn’t expose APIs or ports to the outside world. Using webhooks makes that a little easier, but you still need a system that accepts webhooks and is able to get them across your firewall. That is exactly where Solace PubSub+ Cloud comes in! I’ve built a small webhook forwarder app, that gets data from Solace and sends it onwards without having any of my systems exposed to the Internet.

Trusting Your Ingredients - What Building Software And Cheesecake Have In Common

Mon, 20 May 2019 00:00:00 +0000

At the Twistlock Cloud-Native Security Day, a co-located event at KubeCon 2019, I got to talk about what cheesecake and building apps have in common. As a developer you’re responsible for the security of your app. Security in this case should be seen in the broadest sense of the word, ranging from licenses to software packages. A chef creating cheesecake has similar challenges. The ingredients of a cheesecake are similar to the software packages a developer uses. The preparation is similar to the DevOps pipeline, and recipe is similar to the licenses for developers. Messing up any of those means you have a messy kitchen, or a data breach! In this talk we’ll look at:

Why do we care about licenses?
How does Sec get into the early stages of DevSecOps?
What can chefs and devs learn from each other?

DockerCon 2019 - Persistence Is Futile (Or Is It?)

Tue, 30 Apr 2019 00:00:00 +0000

As we talk to developers we frequently hear the same thing about Docker containers. They’re amazing for managing software, but my app also needs data and configuration. We store those on Docker volumes and we want to reuse those. Now what?

About me

Sun, 13 Jan 2019 00:00:00 +0000

Leon is on a mission to help builders build with data and serverless technologies. As a Senior Solutions Architect at Amazon Web Services (AWS), he partners with customers—from developers to executives—to design scalable, secure, and innovative cloud solutions. His focus is enabling organizations to accelerate their journey with data lakes, analytics, and serverless architectures—turning data into insights that drive real business impact.

He believes that “devs wanna dev.” Leon is passionate about ensuring developers have the right tools, patterns, and guidance so they can focus on what they do best: building. At AWS, he champions customer success by bridging technology and business outcomes, mentoring teams, and guiding best practices to help organizations adopt cloud-native approaches at scale.

How To Use Distroless Containers & OpenFaaS To Minimize Attack Vectors

Mon, 07 Jan 2019 00:00:00 +0000

I’ve been playing with OpenFaas ever since I learned about Minikube a few years ago, so when one of my colleagues mentioned Google’s Distroless project I obviously needed to see if my Go projects could work using those images too.

How To Build A Serverless Contactform With Zeit

Wed, 02 Jan 2019 00:00:00 +0000

Unless you’ve spent the last few months in outer space, or at the very least out of reach from the Internet, you’ve seen that serverless is one of the hottest topics when it comes to building apps. Over the last few months we’ve seen AWS announcing a ton of things at their annual user conference, Google announced support for Go in private beta and serverless containers in private alpha and even Gitlab announced some form of serverless support. With so many massive players it’s easy to forget smaller ones, but those smaller ones are quite often pretty interesting.

DockerCon EU 2018 - The Art Of Deploying Artifacts To Production With Confidence

Thu, 06 Dec 2018 00:00:00 +0000

There are many challenges facing software development specifically when building and deploying new microservices as we try to do every day. Using Cloud-Native technologies we can navigate some of those risks, but not all of our development practices, especially security and compliance, have kept up with the speed in which the rest of our tech stack has evolved. In this presentation I cover how JFrog Xray helps you safely deploy your artifacts to production with full confidence. There are many challenges facing software development specifically when building and deploying new microservices as we try to do every day. Using Cloud-Native technologies we can navigate some of those risks, but not all of our development practices, especially security and compliance, have kept up with the speed in which the rest of our tech stack has evolved. In this presentation I cover how JFrog Xray helps you safely deploy your artifacts to production with full confidence.

Flogo - Event Driven Ecosystem On SE Daily

Thu, 25 Oct 2018 00:00:00 +0000

A smart security camera takes in a high volume of video images and processes those images using a set of machine learning models. Those models can be used to identify interesting snippets of movement throughout the day, and decide which of those snippets to keep. Some of the video snippets might contain movement of birds–but other video snippets might contain footage of intruders.

The Secrets Of Project Flogo - A Deep Dive

Mon, 22 Oct 2018 00:00:00 +0000

Two weeks ago, I had the amazing opportunity to be at AirFrance/KLM in KLM’s Digital Studio to talk about Project Flogo and brainstorm on where they could use Flogo to improve and expand their digital footprint. The team was kind enough to share the recorded video on YouTube!

TIBCO NOW 2018 - Project Flogo Serverless Integration Powered by Flogo and Lambda

Sat, 13 Oct 2018 00:00:00 +0000

No matter the metric, serverless is definitely gaining interest. It’s the dream of every developer, supplying the ability to deploy services in the cloud in no time, automatically scale them, enjoy automagic management by a cloud provider—and, most important, keep it all cost effective! How does this dream become a reality?

This presentation covered what serverless is all about and the benefits of running your apps in the serverless environment. It covers the monoliths-microservices-functions progression and when, where, and why to use serverless architecture and how Project Flogo fits in to the overall picture

Developers, Developers, Developers - Innovating at the Edge

Fri, 05 Oct 2018 00:00:00 +0000

Innovation at the edge is driven by a whole host of people and personalities, but who makes sure those innovations get into production? Developers!

Join Leon Stigter, developer advocate at TIBCO, as he demonstrates tools and technologies to help developers build better software, faster.

This TIBCO Tech Talk covers:

The latest in updates for Project Flogo, an open-source and ultra-lightweight edge computing platform
A brief demo of Flogo and API Scout
How to jumpstart your developer journey

API World 2018 - Project Flogo an Event Driven Stack for the Enterprise

Mon, 01 Oct 2018 00:00:00 +0000

In today’s world everyone is building apps, most times those apps are event-driven and react to what happens around them. How do you take those apps to, let’s say, a Kubernetes cluster, or let them communicate between cloud and on-premises, and how can developers and non-developers work together using the same tools?

The Art Of Open Source Event-Driven Stacks for the Enterprise

Mon, 01 Oct 2018 00:00:00 +0000

Now Where Did I Deploy That Microservice?"

Mon, 17 Sep 2018 00:00:00 +0000

As a developer advocate, I’m in the amazing position to talk to lots and lots of developers. Throughout those conversations I hear a lot of the same concerns popping up. Two of those being, “where did I deploy that microservice?” 😩 and “what is the API definition of that microservice again?"😟

The Art Of Using Go in Flogo

Tue, 28 Aug 2018 00:00:00 +0000

Not too long ago Flogo introduced a new Go API that allows you to build event-driven apps by simply embedding the Flogo engine in your existing Go code. Now you can use the event-driven engine of Flogo to build Go apps while using the activities and triggers that already exist and combining that with “regular” Go code. In one of my other posts, I built an app that could receive messages from PubNub and for this post, I’ll walk through building the exact same using the Go API.

Serverless and Flogo - A Perfect Match (Part 2)

Thu, 16 Aug 2018 00:00:00 +0000

I can hear you think “Part 2?! So there actually is a part 1?” 😱 The answer to that is, yes, there most definitely is a part 1 (but you can safely ignore that 😅). In that part I went over deploying Flogo apps that you built with the Flogo Web UI using the Serverless Framework. Now, with the Go API that we added to Flogo, you can mix triggers and activities from Flogo (and the awesome community) with your regular Go code and… deploy using the Serverless Framework

Flynn in Flight - How To Build Securely Chatting Microservices

Tue, 14 Aug 2018 00:00:00 +0000

I got a ton of great feedback on my post Securely Chatting Microservices, so I decided to create a video out of it and start a new video series called Flynn in Flight!

How To Build Securely Chatting Microservices With Flogo And PubNub

Mon, 13 Aug 2018 00:00:00 +0000

Building microservices is awesome, having them talk to each other is even more awesome! But in today’s world, you can’t be too careful when it comes to sending sensitive data across the wire. Last week I was at PubNub for a Meetup where, together with Jordan Schuetz and Nicholas Grenié, we spoke about cool things you can do with PubNub. One of them is using PubNub as a messaging layer to have your microservices, built with Flogo (duh), talk to each other in a secure way. In this blog, I’ll go over the steps to build those microservices and hook them up using PubNub.

Efficiently Build And Deploy Event-driven Functions to Kubernetes & AWS Lambda

Fri, 03 Aug 2018 00:00:00 +0000

Building multi-platform event-driven microservices and functions shouldn’t drive you mad, and if so, there should be a method to your madness! In this short webinar hosted by DZone, learn to build event-driven microservices and functions using Project Flogo.

TIBCO Meetup 2018 - Building serverless apps with Go & SAM

Thu, 02 Aug 2018 00:00:00 +0000

“Serverless” allows developers to focus on writing their code, and a cloud provider, like AWS, takes care of all the other bits. Building serverless apps means the developer doesn’t have to worry about server management, scaling, or high availability, a convenience that usually comes with the added benefit of lower operational cost. We’ll be showing how to use the Project Flogo lightweight integration engine and open source framework to deploy functions to AWS Lambda using SAM.

How To Build a Slack Bot Powered By Project Flogo

Fri, 15 Jun 2018 00:00:00 +0000

Let’s talk (to) bots! In this case I’m not talking about the awesome autobots, that save the world from Megatron, but Slack bots that help people collaborate and introduce a little fun! Several weeks ago Slack published an article about building the modern collaboration hub and how that hub makes sure that people can interact with each other and their data. On the Internet, everyone loves interaction… and cats, so let’s build a Slack bot that provides your Slack instance with a bit of cat facts!

Why You Should Go Serverless with AWS and TIBCO

Tue, 05 Jun 2018 00:00:00 +0000

Serverless offers huge potential to transform the way businesses build and architect cloud applications. No need to provision infrastructure or deal with maintenance, updates, scaling, or capacity planning - simply upload your apps to Amazon Web Services…

Integration + Cloud - A Match Made In Heaven

Wed, 18 Apr 2018 00:00:00 +0000

We believe every developer has that one special technology they love – whether ESB, open source, or NodeJS. So we decided to put it all together in one easy to use place. That said, you don’t have to use iPaaS to deploy this tech. You can even choose to deploy them on-premises, to a private cloud, to devices, or to serverless environments.

Transform your business app with low-code, APIs, and microservices

Thu, 12 Apr 2018 00:00:00 +0000

Together with the O’Reilly team we did a webinar on Low-Code App Dev. APIs and microservices are awesome if you’re a technical developer, but what if you’re not and you still need to understand how they connect? In this 60 minute webcast, Leon Stigter and Bruno Trimouille of TIBCO Software introduce how low-code platforms can help marketing and sales teams to automate their workflows and deliver on business goals without getting under the hood.

Adopting Serverless Computing with TIBCO & AWS

Tue, 03 Apr 2018 00:00:00 +0000

As the AI-fueled, edge-exposed, blockchain-driven, and streaming analytics-enabled use cases of the future move closer into view, new technologies are needed to make the vision real. Unique and complex workloads accompany the use cases of the future, but luckily, the enabling technologies to compute those workloads have already arrived.

Join TIBCO and AWS for an exciting webinar to help you better understand what serverless architecture is all about, and the benefits of running your apps in a serverless environment. Before you give a listen, how about a quick introduction?

Serverless and Flogo - A Perfect Match

Sat, 24 Mar 2018 00:00:00 +0000

I get to work with serverless microservices on a daily basis, those are services I use myself and ones I help our customers build to take advantage of the benefits that serverless brings you. With many services needing to be deployed and continuous updates, I found myself doing the same thing over and over. It is that specific task that frustrates me most; it simply wasn’t as seamless as I thought it could be.

In this article, I’ll walk you through how I cut the development time and make deployments easily repeatable like a walk in the park — thanks to the combination of the Serverless Framework and a tool called Project Flogo.

A Visual Approach To Building And Deploying Microservices

Tue, 13 Mar 2018 00:00:00 +0000

Together with the O’Reilly team we did a webinar on visually building microservices. Modern and digital experiences today are powered by the fine-grained units of execution we call microservices. However, building microservices isn’t always simple—especially if you’re not an expert developer. How can we make it easy for developers and architects to visually create an API definition so they don’t need to become Swagger experts? Developing your microservices can also be complex, especially when you also need to take into account API specification. How can you visually create your microservices and deploy the same project in a private cloud, a public cloud, and on-prem?

Integration + Cloud - A Match Made in Heaven

Tue, 13 Feb 2018 00:00:00 +0000

Last year on February 14th we published a blog post on “Building the Ultimate Valentine’s API.” Personally, I had a lot of fun writing it and finding facts related to how we spend our Valentine’s Day (though I forgot to bring home chocolates and flowers to my wife, who was none too pleased)! To prevent history from repeating itself, we’re taking things a step further this year with a Valentine’s Day webinar on why Integration + Cloud = A Match Made in Heaven.

Gopherfest SV 2017 - Architectures, Design Patterns, and Lessons Learned

Mon, 11 Dec 2017 00:00:00 +0000

Today I spoke at a Meetup group to discuss the Go Programming Language. Together with an awesome colleague (Miguel Torres) we spoke about Project Flogo and the lessons we learned building it. Click on the link to check out the talk on YouTube.

What Do I Do In Between re:Invent Live Streams? Build Lambda functions

Fri, 01 Dec 2017 00:00:00 +0000

This year I wasn’t able to attend re:Invent, but I did want to do something nice in between the live streams and specifically around Serverless compute and AWS Lambda.

Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume — there is no charge when your code is not running. With Lambda, you can run code for virtually any type of application or backend service — all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability.

How To Deploy Flogo Apps To Kubernetes

Wed, 15 Nov 2017 00:00:00 +0000

With Project Flogo you can visually create Ultralight Edge Microservices and run them anywhere. But what if you want to run those incredibly light microservices using one of the most powerful container management platforms, Kubernetes?

Prerequisites

As described on the Kubernetes website

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery.

If you haven’t set up your own Kubernetes cluster yet, I can absolutely recommend looking at minikube. The team has made an amazing effort to make it super easy to run your own cluster locally with minimal installation effort.

When Is iPaaS The Right Level Of Abstraction?

Thu, 05 Oct 2017 00:00:00 +0000

The explosion of SaaS as the typical acquisition pattern for enterprise software means that any given organization has (or will have) dozens of apps to connect to each other. In fact, the average Marketing department uses about 30 different SaaS apps, and the average HR department…

How To Build Extensions For Flogo Apps In TIBCO Cloud Integration

Mon, 11 Sep 2017 00:00:00 +0000

Last month TIBCO added the ability to add custom activities to TIBCO Cloud Integration — Web Integrator (I’ll use Web Integrator going forward). The Web Integrator experience is “Powered by Project Flogo”, so when you create your own extensions for Web Integrator, and use them in every flow that you want, those activities will work with Project Flogo as well. In this blog post I’ll walk through creating a new extension that connects to IFTTT using the WebHooks service.

How To Build Twelve Factor Apps with Node.js in TIBCO Cloud Integration

Thu, 31 Aug 2017 00:00:00 +0000

Back in 2012, the engineering team at Heroku created a set of best practices to develop and run web apps. That document, consisting of 12 incredibly important ‘rules’, was dubbed the 12 Factor App manifesto. Over the years the document gained a lot of traction and especially with the rise of microservices having a 12 Factor App compliant app became important. With the rise of microservices a lot of other practices and tools (like git, DevOps, Docker and Configuration Management) became very popular as well.

What Every Node.js Developer Should Use To Deploy To TIBCO Cloud Integration

Wed, 30 Aug 2017 00:00:00 +0000

Pretty much all the large cloud platforms provide not only a great visual interface to get things done, they also have a great command line interface. As much as I love a great UI when browsing the web, I do tend to favor the command line interfaces when I’m focusing on developing things.

How To Continuously Test Flogo Activities With Jenkins

Wed, 23 Aug 2017 00:00:00 +0000

In 2016 TIBCO announced Project Flogo as an ultra lightweight integration engine. It is the lightest way to connect IoT devices being up to 20 to 50 times lighter than Node.js and Java® Dropwizard. Project Flogo was made available as Open Source Software because TIBCO believes there is a better way to integrate IoT devices. With it being Open Source and easily extensible, you want to make sure that the activities you build remain working after each check in on GitHub. These are a lot of words to say “How can I test my activities every time code is pushed to Git”?

How To Add Basic Auth To Flogo Apps in TIBCO Cloud Integration

Mon, 21 Aug 2017 00:00:00 +0000

Over the past few weeks I’ve gotten many questions on how to use Basic Authentication with the new Web Integrator in TIBCO Cloud Integration. It is really easy to do that!

How To Monitor Your Node.js Apps With Prometheus"

Fri, 18 Aug 2017 00:00:00 +0000

In the age of large monolithic apps and app servers, it was not too difficult to monitor the status of your application as well as some more detailed monitoring of transactions. In today’s age of microservices, that whole monitoring thing is getting more and more complex. You not only have to deal with a bunch of more servers, you also have to deal with more (micro)services. You have different options depending on what you want to monitor. For example, you can use Nagios, Zabbix or Prometheus. My personal preference goes out to Greek deity that stole fire from Mount Olympus and brought it to us.

How To Combine APIs With Flogo Apps In TIBCO Cloud Integration

Wed, 16 Aug 2017 00:00:00 +0000

In 2002 Jeff Bezos issued a mandate that would change the world forever. At the very least it brought a massive change to how data is reused on the Internet. The mandate from Jeff Bezos was:

All teams will henceforth expose their data and functionality through service interfaces.

Teams must communicate with each other through these interfaces.
There will be no other form of inter-process communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.
It doesn’t matter what technology they use.
All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.
Anyone who doesn’t do this will be fired. Thank you; have a nice day!

That mandate started a great deal of what we see as the API economy today. Many of the worlds’ enterprises have APIs available that can deliver data so you can focus on building value, rather than worrying about how to get the data. Having said all that, most APIs that are out there are documented but don’t have a swagger.json that you can import in your code. The Web Integrator in TIBCO Cloud Integration gives you the ability to copy and paste the sample messages that those APIs do have and use that as the base to invoke REST APIs.

The Art Of Building REST Services in TIBCO Cloud Integation

Wed, 16 Aug 2017 00:00:00 +0000

You shouldn’t have to be a Swagger expert to get started with designing and building your new API. Creating an API from scratch can be a difficult task, so what if you could create API without writing a line of code?

How To Connect Google Forms to APIs

Fri, 11 Aug 2017 00:00:00 +0000

Have you ever had that feeling you wanted to capture data and just send that data to just about anywhere? Google Forms gives you an easy way to create personalized surveys with the style that you want. So we got the first requirement down with that one, but what about sending the data elsewhere? That is where TIBCO Cloud Integration comes in.

How To Send Emails Using Flogo Apps in TIBCO Cloud Integration

Tue, 08 Aug 2017 00:00:00 +0000

Communication in our daily life is incredibly important, so why should that be any different when you’re building integration processes? Sending emails is still considered an important part of day to day operations. For example when errors occur in your integration flows, you might want to send an email. In this tutorial I’ll walk you through sending an email, and in my case I’ll use Gmail as provider.

How To Use Git Hooks To Automatically Deploy Apps

Mon, 07 Aug 2017 00:00:00 +0000

Probably the most common version control system, used by developers today is git. Whether that is a self hosted server (like Gogs), a bare repo (git init) or with GitHub, most developers intuitively seem to choose for git as their version control system of choice. I try to store all my projects in local git repos and some of them make it to GitHub, while many of them don’t. When it comes to deployment of apps to TIBCO Cloud Integration, I do many updates per day so I wanted an easy way to not only store my latest source but deploy them right after.

How To Set Up Continuous Integration with Jenkins and Node.js

Fri, 04 Aug 2017 00:00:00 +0000

As you’ve probably guessed by my previous two articles I love writing (about) Node.js. What isn’t great, however, is creating deployment artifacts every time you check something in to GitHub and want to deploy. To make that process a little easier there are many tools available, and of of those is Jenkins, the friendly butler. In this “How-to” I’ll walk you through using Jenkins with the tibcli utility to deploy Node.js apps to TIBCO Cloud Integration every time updates are pushed to GitHub.

VSCode Extension For TIBCO Cloud Integration

Fri, 04 Aug 2017 00:00:00 +0000

I’ve just updated the Microsoft Visual Studio Code extension to help develop and deploy Node.js apps to TIBCO Cloud Integration. Apart from a whole bunch of restructuring, it now has the ability to create a new Node.js app (if you don’t want to start from an API spec) and it makes use of the .env files to work with process.env context.

How To Use Express Middleware To Filter Traffic In Node.js

Thu, 03 Aug 2017 00:00:00 +0000

With the introduction of Node.js in TIBCO Cloud Integration you have an amazing toolset to connect to almost anything and build APIs. In fact, there is a good chance that if you know Node.js that your first application was a simple application that said “Hello World” every time (your first API!). Let’s create a custom Express middleware that checks if the IP address of the sender matches a predefined list. In this tutorial we’ll use the list of TIBCO Mashery Traffic Managers as a ‘whitelist’ (so traffic from all other IP addresses will be blocked).

The Art Of Getting Back To Your Data Securely!

Wed, 10 May 2017 00:00:00 +0000

If you are like me, the data I need to do my job exists not only in the cloud. It can be hard to get to all data sources, especially when those are on-premises and behind a firewall. I am not alone, as pretty much everyone is facing these challenges. In fact, Gartner predicted that over sixty-five percent of all integration flows will be created outside of the control of IT departments as a result of the growing number of integration related tasks that they need to take care of. Simply put, organizations today are integrating to everything. The ‘everything’ in the last sentence not only includes Software-as-a-Service applications like Salesforce.com or NetSuite, but also includes applications and services hosted in private networks and datacenters.

The Art Of Building Node.js Microservices in TIBCO Cloud Integration

Wed, 05 Apr 2017 00:00:00 +0000

The world of integration is hybrid. Not only hybrid in the sense that you have on-premise and cloud-based applications, but also hybrid in the types of people that connect systems together or build something completely new. What really doesn’t change is the fact that people want to use the tools that fit their purpose.

There is quite a good chance that you know Node.js. According to Techworm, it is the number 7 programming language. If you’ve ever built a Node.js app, chances are pretty good that your first app said “Hello World” every time. In fact, that might even have been your first API!

The Secret Of The Ultimate Valentine’s API

Tue, 14 Feb 2017 00:00:00 +0000

Integration is red (it is my heart, after all), clouds are blue, interconnect everything and I’ll 💙 you!
With the theme of TIBCO NOW this year being “Digital Smarter”, I wanted to see if I could build the ultimate Valentine’s Day API using our own technology while considering the requirements that might impose on one’s choice of tech.

Valentine’s Day is traditionally the holiday where people receive cards from their significant others and secret admirers and is also a great day to have a first date. What to do on a first date? Catch a movie. According to research, one in ten people would ask someone out based on their movie preferences, so having the ability to connect to different film APIs could make or break that first date.

How To Create An API Without Writing Any Code

Wed, 14 Dec 2016 00:00:00 +0000

To successfully compete or even survive in today’s ever-changing business climate, organizations need to become more agile. They need to respond to customer expectations and market changes more quickly. Companies are doing this by using and building APIs. APIs are the model for quickly building and growing successful businesses. The Internet has transformed from a network of informational web pages to an ecosystem of APIs and applications that work together to empower new applications, new businesses, new ways of working together, and new business opportunities.

CloudExpo 2016 - The Road to a Cloud First Enterprise

Tue, 01 Nov 2016 00:00:00 +0000

Companies must find a way to join both paths and view the transition to digital as a unified journey, with the end goal clearly defined, then utilize APIs to help them get there faster. The question then becomes, how can companies and developers leverage ESBs, APIs, and a Fast Data platform to cultivate innovation?