The past few weeks I spent building a front-end for an app store crawler that I made. The end goal is just to provide an open front-end for the apps details for the approximate 2.5m apps (already 1/2 those are no longer live) I’ve crawled from the Google & Apple stores. The site is now live on appgoblin.info and hosts a variety of Android and iOS stats and store information. Hopefully it’s something that some marketers may find interesting, and the code is all free on Github.
History
Originally this came out of a curiosity I had about the IAB app-ads.txt standard in 2022. In order to scrape app-ads.txt files I had to first get all the iOS and Android apps + their respective publisher URLs. This ended with a small dashboard I put on https://ads.jamesoclaire.com/dash/ads but I never expanded it further.
While that project, like app-ads.txt itself, went unused, I remained interested in letting the database of apps keep growing.
Then in the fall of 2023 while on hiatus from real work, I started working on an API and UI. Creating the UI gave me exactly what I wanted, which was use cases for displaying the data. Adding one feature led to the next so now there are several sections to browse.
Features
App Details
Each app has it’s own page for all the data pulled from the app store as well as second order of information derived from that data such as installs per day or change in ratings over time.
App Rankings
Since we have each apps history, it’s easy to build the historical ranking charts. In this example on the right, you can see a game quickly climbing multiple charts at once over the past month. This change in rankings is a great way to track an apps popularity over time.
Ranking Charts
No site like this would be without it’s own app rankings page. This is straight from the Google & Apple stores top charts each day for tracking apps overtime.
New Apps
Now this might seem like an obvious one, but it really brought home the slow burn that the app stores barely surface new apps anymore. Long gone from both stores are any structured “new” sections which showcase new apps. The cynical answer here is that those sections actively take away from the ad-revenue generated by the in store advertising that makes up a large percentage of the apps on screen at any given time in each store.
So, why does AppGoblin exist?
Yes, there are a large number of similar offerings like data.ai and sensortower.com which have many more features. For me, I enjoyed learning more front-end and Javascript as well as give me something to do with the scraper which I had built previously (whats the point of a database if you cant use it somehow). For now App Goblin is simply here as a resource free of charge as it’s running costs only $20 a month or so in hosting.
Open Source Tech Stack
First is the underlying data which is collected and maintained from github.com/ddxv/adscrawler The adscrawler uses Python & PostgreSQL to crawl app stores to collect apps and their basic information like names and rating counts. This data is then stored to the PostgreSQL database. I also use some of the original parts of github.com/ddxv/app-ads-dash for monitoring the scraper. Hopefully in the future the parts specific to the scraper are moved to that repository.
The website itself is also open source and can be found at github.com/ddxv/app-store-dash The website is built from a Python backend API based on LiteStar which manages queries to the PostgreSQL database. The API is used by a JavaScript Svelte frontend with Tailwind and Skeleton for CSS. All projects are glued together with Systemd and web sockets where needed. Everything is hosted on AWS in the small EC2 instances.
As always, if you have any interest or questions please feel free to reach out.
I love using widgets. I probably have too many of them on my phone’s home screen. News apps, weather apps, stocks, stats. I love being able to glance through them without opening the corresponding apps. This led me to realize though that I have seen very few widgets for mobile games. As I started working on making widgets, I realized you could actually make simple games inside of widgets.
Many apps use widgets already, but I think due to poor tooling, not many games have embraced widgets as a part of their game play loop. Recently I started looking into what some of the possibilities around making a simple game which could be played inside of a widget.
Widget Limitations:
Very limited dragging or other complex touches. This means no drag/swipe functionality. You can implement drag for lists, but picking up an item and dragging it around seems out of the question.
Animations are quite limited. I don’t think implementing most game animations would be possible.
Computations are subject to being cut off at any point. The widget must essentially be at rest at all times, this definitely limits what you can do, but still fits well for some gaming uses.
So, let’s try a Tic-Tac-Toe Widget
I chose Tic-Tac-Toe since it was the game I could think of with the least logic. My best guess was that implementing the game logic might be tricky or troublesome. Additionally, the turn based logic was perfect for falling into the background when the game is not being played.
This turned out to be pretty easy. I was able to get a prototype up and running within a day or so. One surprise I had was having trouble using a delay. In the end I gave up trying to implement a half second delay to simulate the computer ‘thinking’. I’m not sure if that is something that could be addressed later or not.
The game is mostly limited by the fact that it’s just tic-tac-toe that you’re playing, which gets old fast.
What about a more difficult puzzle game?
Next I went with a simple sliding puzzle game. The goal is to slide the pieces to slowly sort them into the correct order. These could be a puzzle image or as you see on the side just pure numbers. This would definitely be a bit flashier if I used images, and I think that is also doable for the future.
Though this is barely more complex than Tic-Tac-Toe the difficulties became apparent quickly. The biggest issue I faced was that the Android operating system will randomly shut down the widget’s thread which would cause the game to lose anything currently stored in memory.
This means that the game state can’t be stored in memory, but needs to be backed up to disk each time, then recovered. So each ‘turn’ is played by the user, then the game state is saved to disk, and when the widget is interacted with a second time, the game state is read back from disk first.
This means the game is quite solid and recovers from it’s process being ended without the player knowing, but it is just a bit laggy when ‘playing’ quickly as you can see in the gif.
That’s cool, but what about non puzzle games?
This is where I started getting more excited. While I think you could recreate many simple puzzle games as widgets, what is the point for the developer? While the player might enjoy some play outside the app any of the more pleasant animations or clever game play mechanics would remain out of reach for the game widget.
But what the widget does give the developer is a large real estate for the the player’s home screen. This could be a place for managing notifications to entice the player to open the game. A few examples:
Connection to in game functions like responding to events or setting a status
Notification that an in game timer is finished
Countdown to live events
Button to collect daily reward and open game to boost game re-engagement
Community Bulletin board
I think the ideas here are pretty basic, but could really help to reconnect games with their existing user bases.
If anyone out there has any interest in trying out the idea of building a widget for a game I’d be happy to help.
If you’re curious about either of the above examples you can get both on Google play and all code on GitHub:
Hacker News is one of my go to sites to read the best takes on the latest tech news.
Probably the reason I keep going back to it is the honest discussions and supportive community. But you probably already know this, since Hacker News is one of the most popular tech news sources out there.
And though there are dozens of high(er) quality Hacker News apps out there, I’ve always wished they supported widgets. In fact most apps I use I wish they supported Widgets.
I love waking up in the morning and checking the latest news from my home screen, rather than opening up web pages. This is why I decided to use Hacker News to make my first Android app and widget.
The whole experience was absolutely amazing compared to my recent attempt to use Unity. Android Studio + Kotlin and Jetpack Compose were an absolute dream to use. I rarely felt frustrated and was able to envision a few more interesting projects using widgets coming up next week.
In the meantime, please feel free to use my app or code:
It certainly isn’t anything amazing, but it was something that I really wanted to do for myself.
After recently stepping away from my position at Bubbleye and taking a month or two to enjoy some camping, family time and generally enjoying life I finally got back to Taipei and realized I had a unique opportunity to work on side projects that I’ve always wanted to do. Top of that list was to make a simple strategy game to better understand game design and the monetization side of games. To that end I built my own mini-RTS game using the game engine Unity.
Though Unity is the most popular choice for indie devs, I had an another motive: to learn more about Unity’s ads monetization side, especially as things change through the acquisitions of ironSource and TapJoy (originally acquired by ironSource).
How to make a good game? Be good at everything
I consider myself a generalist, competent at many things and flexible when learning new skills, but not necessarily skilled in all of the things it takes to make a game. There are so many integral parts for games like game graphics, game design, computer science and UI/UX, all of which need to be put together with good game design. It was a taxing job to work on build all these parts, and in the end also try to make a game design loop that was fun. I think this was something that surprised, because as I said I thought my strength would be that I can wear many hats, but making a game you REALLY need to do everything well, and missing even little parts of the loop made it glaringly obvious… the game wasn’t that good. As someone who enjoys the technical problems, it was easy to skip working on the game play loop itself, so I gained respect for game designers who do have the tenacity to tweak their game again and again to find out what would make a fun game.
So what did I accomplish? I learned a lot about Unity
Well, I learned quite a bit of how difficult Unity has become to use. Unity is bloated and was very painful to use. Many times, difficult problems would come down to conflicting settings within Unity itself. Warring Unity features which seem at odds with each other, and the only way to find out was to tap or untap another checkbox. The UI settings and features in Unity feel like a huge spaghetti of new features from last month and legacy support for systems from 10 years ago. I think what was frustrating is that unlike a true IDE, your code barely matters, and the Unity UI components often have little warning or information for what they might do.
This past month news broke that Unity’s pricing model was changing and the uproar was everywhere. This is fueled by a general distrust of Unity. I think it’s interesting, because if I look back, I certainly didn’t feel much trust in Unity’s product. Additionally, the code that is written, is so tightly bound to Unity it is impossible to move it to another Engine without simply rebuilding the entire game. Finally, I think that generally the state of mobile ecosystem is such that another middleman, no matter how respected, trying to assert it’s position to take a revenue share struck a nerve with many people.
Next
Having my first game in some ways did make me excited to make another. Parts were quite fun, but using Unity was a serious pain. I realized this when after the game I moved on and made a couple Android Apps in the weeks after using Android Studio. It was amazingly easy. The contrast of easily building apps in Android studio vs Unity’s spaghetti mess of code, buttons and packages that only maybe co-exist with each other was incredible.
Still, having my own apps is teaching me more about recent changes in the ad monetization landscape.
Also, having my own game gives me a good chance to work on more man in the middle attacks to watch mobile HTTPS traffic in and out of app and app SDKs. This is something I’m quite interested in, and being able to have a better understanding of how apps are built in 2023 is very useful.
These are my notes as I try to find a reliable way to collect HTTPS traffic from mobile apps on my phone or VM. Still have some questions as to which works best. This is all very much just a WIP / notes, but feel free to add or use. This took quite a bit of trial and error with a number of not working solutions, until I found this recent comment which worked perfect. Incase this helps anyone else I’m just writing my process here. Feel free to comment or let me know if you have other advice!
=====================
Waydroid VM & mitmproxy Setup Notes
Waydroid and mitmproxy are the two main tools you will use. Waydroid is an emulator for Android on Linux and will need a variety of custom software installed in it to make it work.
Ensure iptables is working. I was able to solve this by explicitly add /lib/x86_64-linux-gnu/xtables to /etc/ld.so.conf.d/x86_64-linux-gnu.conf and rebooting. This was reported working for Debian 11 and Ubuntu 20+
Setup iptables. I put the necessary iptable additions into a script as I ran them quite often, and can sometimes pause your local or Waydroid connection and needed to be cleared after using. You can use this script or copy paste the code block below and adapt as needed.
Run ./proxysetup.sh 8080 -w for waydroid. Use -l if mitm for other device
NOTE: proxysetup.sh runs sudo iptables -t nat -F at end to clear out iptables. This is because some of the iptable settings depending on proxy type can cause your connection to be blocked. But be warned, this will clear all custom iptables on your nat table you may have added.
proxysetup.sh runs the following commands, so feel free to run them yourself: ```#!/bin/bash sudo iptables -t nat -A PREROUTING -i waydroid0 -p tcp --dport 80 -j REDIRECT --to-port $port sudo iptables -t nat -A PREROUTING -i waydroid0 -p tcp --dport 443 -j REDIRECT --to-port $port sudo ip6tables -t nat -A PREROUTING -i waydroid0 -p tcp --dport 80 -j REDIRECT --to-port $port sudo ip6tables -t nat -A PREROUTING -i waydroid0 -p tcp --dport 443 -j REDIRECT --to-port $port sudo sysctl -w net.ipv4.ip_forward=1 sudo sysctl -w net.ipv6.conf.all.forwarding=1 sudo sysctl -w net.ipv4.conf.all.send_redirects=0 mitmweb --mode transparent --showhost --set block_global=false ```
Start Waydroid service: waydroid session start
Start Waydroid UI: waydroid show-full-ui and check internet
Inside Waydroid, if internet can’t connect try sudo waydroid shell and check ip link and ip addr to see if firewall blocking. More info at: ArchWiki Waydroid Networking
LSPosed Zygisk module trying to provide an ART hooking framework which delivers consistent APIs with the OG Xposed, leveraging LSPlant hooking framework. We will use this to install SSLUnpinning in a future step.
Finally, we can install our two custom tools into Magisk & LSPosed respectively:
MagiskTrustUserCerts This Magisk module will take your user CA certs and move them to system or ‘root’ CA certifications which more apps will trust.
SSLUnpinning This LSPosed module helps to unpin apps during runtime.
Once everything is installed, shut down and open Waydroid and mitmproxy one more time. After this you should be able to see clear text HTTPS requests from your Waydroid VM.
========
Other Tools
Emulator vs Phone
This is the first question and probably the most dependent on what you want to achieve. Working on a real device gives more space between your device and the proxy which makes things easier. The extra space is costly in other ways. For example, I would prefer to have a single instance running on the computer to collect information, but using a phone is easier but has the physical requirement of a device connected to the network.
Phone
Physical separation allows for clearer testing. Fully functional device means your input and output work as expected.
Emulator – Waydroid
Emulator running on the same computer causes more complicated networking to ensure you don’t block your own traffic. Troubleshooting is trickier as it’s more difficult to easily access parts of the emulator that a phone is easy to access. For example, I spent much more time than I would have expected to move a VPN configuration file from my computer to the virtual machine emulator than I would have ever expected. Adding the same configuration to the phone was a simple QR code scan.
Emulator running in a virtual machine allows for a future use case of running the whole thing in the cloud without a physical device.
Proxies
As far as I know, the only way to capture the HTTPS traffic is to use a proxy. This is in the form of an application running on a separate (virtual or physical as mentioned above) device. The hardest part here is the Certificate Authority which signs the HTTPS traffic when it leaves the app. More sophisticated apps, to prevent fraud, do a variety of actions to prevent the user or 3rd parties from capturing the data in each HTTPS request.
I tried this first as it comes with Python library which would make capturing data for later analysis much easier. Mitmproxy has a few different modes, and ultimately I found that mitmproxy --mode wireguard which runs via VPN captured a good amount of traffic, but still had target SDK traffic unable to be opened. Mitmproxy has a built in tool to help installing the certificate in Android as a user certificate. This will capture some HTTPs traffic, but for some apps and many SDKs this does not capture their traffic. Traffic can be captured in several ways: CLI tool for analysis of live traffic in memory, CLI dump to file and in memory live in browser of choice.
I first used Charles nearly 10 years ago, and it doesn’t feel like it’s changed much, but is actively maintained. When I first started using Charles it was a breeze to use, CA was less of a problem. But as Android changed it also now has the problems of CA needing to be installed, and helps the user by providing it’s own signed certificate which can be installed as a user certificate. Charles is a standalone program that you run and as such it does have a fair amount of issues on my linux environment related to it’s display sizes.
Community edition that is free to use. Runs in browser and comes with it’s own CA tool.
Android Certificate Authority
These are the certificates used to sign HTTPS traffic to keep it secure. In Android there are three levels: User, System (root) and App Pinned Certificates. In Android settings you can add a CA which will be considered “user”. Apps can choose whether to ignore this certificate. System CAs can only be set by a root user. While a user can install user CA’s, apps do not have to use these. CAs can be set by users as root certificates. I believe this must be set regardless of device or VM. The majority of the certificates provided by the proxies don’t seem to open a lot of HTTPS traffic. This is likely because Android N (API level 24) certificate pinning was introduced in 2016 and at this point most SDKs and Apps use this for transferring traffic.
This is installed on a device or emulator. An Xposed addon that can be installed to force apps to use root authorities and prevent them from pinning their own CA.
This can be installed in a separate linux environment and is used to modify an app’s apk before being installed into a VM emultator or phone. It attempts to get around the app’s certificate pinning by patching the APK to disable certificate pinning.
Looking around app-ads.txt files one of the first things I was excited to check was whether DIRECT publisher IDs show up on other unrelated apps. To do this I ignored any apps that shared the same developer contact URL or Developer IDs. This doesn’t catch 100% of the legitimate publishers I checked, as the examples to follow show, but does provide a good rough estimate of which networks seem to have publishers whose IDs are more mixed.
DIRECT traffic should mean that your app controls the publisher ID listed. But sometimes when looking through ads-txt there are publisher IDs that appear many time on unrelated apps. This could mean that an ad buyer could be buying ads from more than just the app they were intending, which means both good and bad quality could be mixed together.
The worst possible use case is a bad actor ad network mixing unrelated publishers together, then asking each of them to use the DIRECT tag. It turns out this is much less obvious than I first assumed, but it does seem like certain networks do have a higher prevalence of mixed IDs. Let’s look at some examples.
Number 1: Quiz Game(s)
One of the first examples I looked at was Blackpink Quiz Game who’s Unity, Google and other publisher IDs were shared across hundreds of other apps like this Family Guy-Quiz. As you can see, the two titles visually look quite similar. Each of these hundreds of apps belonged to different Google Play developers each with their own developer ids.
In addition to being visually similar, they also the same top level domain hosting their developer page url. For example Blackpink: http://pub-nbzaw-b7jg6.quickappninja.com/ and Family Guy-Quiz: http://pub-9j4jn-zu5k3.quickappninja.com/
Each link does host it’s own app-ads.txt file, but they were nearly identical across each app, containing the same DIRECT and RESELLER publisher IDs. Looking at the top level domain we see that QuickApp is an online tool that lets you create quize games from your browser. When you export the APK from the site it would likely all contain the same ad monetization accounts, controlled by QuickApp. QuickApp does later share some of the ad revenue back to the original creator.
To sum up, while that is an interesting business model, it certainly is not the kind of ad fraud I was originally looking for, and for now I’ll leave this as undecided for how to handle it.
Number 2: Crazy Marvin
Next try found another Unity ID that showed across a hundred plus apps. Checking the apps I didn’t immediately see a pattern this time. One a Dragon Ball Z game by Yodo1, a large game publisher, while the others were smaller other games with different styles and developers. Most of the developers did have several games.
I found it really interesting that the oddest of the titles, Pimple Popper, had a GitHub linked with it’s game open sourced with a group of people working on it. So I reached out to Crazy-Marvin the dev of the Pimple Popper game to see if he could lend some insight.
Marvin was very helpful and wrote back immediately to let me know he didn’t know much about ads, but that he had been contacted by Yodo1 and per their request put the relevant ad SDKs and app-ads.txt file on his site.
Again, this seems to be a monetization partner who runs ads on behalf of the client, but without taking over their developer page or showing any other obvious links.
Finally I found one that looks more suspicious than the others. This ID 24400 is located in 500+ separate developer sites meaning 1000s of apps are pointed to this publisher ID as DIRECT traffic. Oddly, it is almost always paired with a RESELLER line with the same ID. RESELLER here would be fine, as it is being resold, but DIRECT seems highly unlikely. Looking through the apps again they all appear to be unrelated.
Compared to the first and second attempt, Rubicon is not a SDK ad network, but a programmatic SSP.
Outcome: Inconclusive, but a picture is starting to form
While there is a lot of different ways we can put together our monetization accounts, there do seem to be a higher prevalence of mixed/shared IDs on the programmatic only traffic such as Rubicon, PubMatic, IndexExchange. Looking again at our plot from the beginning, highlighting the traditional networks that do both buying and selling, we do see that they tend to rank higher than the pure programmatic SSPs.
Next?
I think that the results are pretty inconclusive, but do show some directions to move forward. I learned that there are quite a few situations where shared DIRECT publisher IDs is a natural outcome of some company’s revenue setup, but still some situations appear to be incorrectly labeled publisher IDs. The networks that do have these shared publisher IDs seem to be more prominently programmatic SSPs.
Do you have any specific examples you’d like me to check? Feel free to comment or send me your developer URL and I can help look into your IDs. Do you think there is interest in a tool to help check your app-ads.txt for network quality?
Google’s market share in in-app advertising is unmistakably dominant. The data above was scraped from 100k+ Google Play Store & Apple App Store apps (with a slight emphasis on games) who use advertising for monetization. This public data is made possible by adoption of IAB’s app-ads.txt standard which allows buyers and sellers to cross verify their ad buying is coming from legitimate direct sources by checking public txt files.
So, Why is Google So Dominant?
First, let’s confirm that the data is somewhat accurate. This data can be correlated to similar reporting by Braedon, creator of well-known.dev, a repository of public data from app-ads.txt files (and much more). At 92% his percentage for Google lined up perfectly with the data here, despite likely slightly different app populations.
Amongst app-ads.txts on the top 100k domains, Google is actually slightly higher at 91% (89% DIRECT).
Magnite is once again next, with 74% (58% DIRECT).
His tweets were also picked up a few weeks ago by AdExchanger, who pointed out that as you look closer at the long tail of small advertisers, the app-ads.txt files often contain only one single line: Google.com
Why is Google dominant in mobile? As the operating system for Android, they are the easiest way to integrate an advertising SDK for a small app. This means that the first ads shown, are often through Google.com and if and when the app grows, they will only ever add more competing ad networks, likely never removing the original option. Thus Google is the default advertising option on the device, with other ad networks barely getting a second thought.
Is There No Competing?
Let’s zoom in a bit on the data, using only the top 5% of apps ranked by install/review count, and we see a slightly more competitive world. Here Google is still on nearly every app, but the next closest ad networks are on nearly a 1/3, a healthy increase.
This seems to indicate that once an app a big enough monetization team to manage more than one network, they can much more easily expand to other networks so as to increase their eCPMs. Adding the first ad SDK is a pain, and Google’s moat around their advertising castle is that adding and managing the second SDK takes serious commitment and cost.
What else are we missing?
Let’s start with a caveat: this is programmatic traffic, measured only by those who are hosting their app-ads.txt file. Some ad networks are not so transparent.
For example, Apple Search Ads, with it’s growing stranglehold over the iOS advertising ecosystem is entirely absent in app-ads.txt. This is because Apple does not compete at all in the programmatic markets, but instead uses it’s elevated role as the owner of the App Store to manage it’s own ads.
Another missing piece is that the vast majority of app-ads.txt are polluted now by traffic marked as resellers. Resellers are much more competitive market, fighting over the scraps of what is left after the top networks take their pick of traffic. Unfortunately, there are so many resellers, it is quite hard to understand how app-ads.txt helps with verification at all, as I have seen many app-ads.txt files with hundreds of resellers and only 1 direct account, usually just google.com.
Closing thoughts
Despite the doom and gloom, the most important thing is that this information is publicly available. Advertising is a difficult to parse business and this helps us to have some transparency on the marketplace we all share.
Disclaimer: All images and ads in this post are for spec commercial examples only. The originator of the commercial is not affiliated with, connected to, nor sponsored or endorsed by Clash of Clans (or Starbucks).
TLDR: Fully formed art, but difficult to control details
AI Text to image models have seen explosive growth the past year matched with an equally explosive growth of an assumption that AI generated images will replace graphic design and artist roles. I am not a graphic designer but in my role of developing advertising automation solutions for mobile games, the creative development process is something I’ve worked near for years.
Input: “A steaming white coffee mug of cappucino with the (starbucks logo), photorealistic”
In short the AI image creation goes from nothing to fully formed image with no effort, but specific inputs or requirements increase the difficulty immensely. AI image generation is the perfect tool for creating a never before seen magical world, but if your requirements are more specific than that, it can feel quite difficult to impart your intentions into the chaos of the random image generation.
For example, this generated photo of a Starbucks coffee mug logo is OK, but the mermaid logo’s face is slightly odd. Further steps can get it closer to the original logo, but ultimately it never will be the exact right logo or the exact right color green, because those requirements ultimately comes from the brand creator and cannot be randomly generated by the AI and would instead later need to be added.
The setup and tools: Stable Diffusion
For our imaginary use cases we will use Clash of Clans as our example spec advertisement. Clash of Clans is a popular mobile game with a proliferation of content, so the image databases used to train the models already have a fair amount of tagged image content. Stable Diffusion is built on models based on 5B images, scraped from the internet, and fully searchable on LAION-5B: Searchable Database of 5B Tagged Images. Stable Diffusion starts from a random noise pattern (like static on a TV) then iterates until patterns begin to form that match your text or image input.
I ran all these images through the Web UI provided by AUTOMATIC1111, this UI allows you to run edit and play with Stable Diffusion in your own browser. The images were run on a regular laptop with 4GB GPU memory, so all images were output with the bare minimum settings. Of course, if you have the recommended 12GB GPU memory your outputs should look much better. In general I found that the iterative approach worked best, so was able to break the project into smaller steps which usually fit nicely into my 4GB memory.
AUTOMATIC1111 Web UI with convenient sliders, buttons and toggles for Stable Diffusion inputs
Use Case 1: Resizing an Ad from Landscape to Portrait
A common situation marketers encounter is after a set of creatives are created, at some point in the future they may encounter an advertising platform with different dimension requirements.
To start the following creative is 1140×700 pixels but we need to transform it to 900×1600 pixels. This is quite difficult as you not only need to resize the image, but also fill in the additional empty space. Stretching the image would distort the creative to the point of being unusable, putting in blurry backgrounds is another simple fix, but still an aesthetically lacking option.
First Try: Img2Img
Using Img2Img with a prefilled image will set a tone for the image to be generated. I tried several rounds of pure Img2Img, and while some basic contents were similar (note the characters on left, white light background on right) the content was so wholly different that it invalidated the original use case to use the original art. I want to keep as much of the original art as possible, so this doesn’t cut it.
Input: “mobile fantasy game battle”“cloudy sky on a green grassy hill with arrows flying in style of a clash of clans mobile game”
Step in the right direction: InPaint
Using Inpaint and a custom black and white image, both made in a separate image editor.
Input: “clash of clans mobile game ad rushing towards battle cry in the style clash of clans”
In our next attempt, we use InPaint which has the tool for creating a mask (the white and black part), which select which parts of the image should remain original and which should be generated. In this case, the white and black below correspond to the generated and untouched parts of the image respectively. The remaining image (blue sky / some of the original image and green for grass) are then used for Img2Img inputs.
This gets us much closer to something that looks OK. We have a purple character above and a semi-convincing charge of men below. The line between the images does not blur very well and the their body positions are illogical. The entire scene at the very bottom is the worse as the men closer to monsters than Clash of Clans characters.
Keep the image: Fix the remaining issues
We can try to fix these by further rounds of InPainting, focusing on specific sections. For example, let’s remove the purple sky man in favor of clouds.
Img2Img Inpainting over the giant purple face, creating clouds instead.
Trying to get the faces to be more “on brand”
The two faces in the bottom of the image were still bothering me as they didn’t quite look like the Barbarian character from Clash of Clans. The main issues were the lack/odd eyes and non yellow beards. What initially felt like a quick fix turned into a much longer process of trial and error, generating dozens of attempts.
Inpainting the terrifying faces away
Finally, I tried searching through LAION-5B database, which was used to train the Stable Diffusion model. I found that probably the best input for getting two good Clash of Clan’s barbarian faces was to simplify my input to “clash of clans icon yellow”. In the search on LAION-5B database this returned the highest density of typical Clash barbarian characters.
LAION-5B search for clash of clans
Use Case 1: Final Resized Image
After getting the last two faces’ beards fully yellow, I had arrived at what my dwindling patience told me is the final product. The outcome is something only a parent can be proud of, and while interesting, would not be something that any branded advertisers would be comfortable using as an advertising creative.
As close to final as I could get
The difficult in controlling the coloring and themes leads to character faces are very similar to Clash of Clans branding, but ultimately not on brand.
A higher order issue is that while we did solve our initial problem to resize the original image, we did not do it in a way that is flexible for future changes. The end product is a fully rasterized image, which means that it is even less flexible were it needed to be further resized for another set of dimensions. A typical work file from a creative team would include each section or character in their own folders and layers, allowing for quick modifications later.
A time breakdown of generating this image was as follows:
Setup: 0.5hr
First images to the clouds: 1hr
Generating the two faces: 1.5hr
Overall generating this use case felt like it became a slower and slower process as I tried to exert a high level of control over the image being output.
Use Case 2: Themes
Sometimes for holidays you may want to modify existing creatives to add a holiday theme. This is usually a pretty simple process for creative teams, but for smaller companies there might not be enough time or manpower to dedicate for creating a steady stream of themed content. Let’s see how well we can take our original creative and add a Halloween theme. Let’s start with some fall colors and see what we get.
And now let’s add some pumpkins!
Losing the plot, trying to get more of the Halloween theme
Trying to take the Halloween theme further it is really easy to quickly get off track. The image below is just an example, but most images quickly become 100% unrelated to the original image. Again this underscores the true power of Stable Diffusion: that it can so quickly generate incredibly unique and interesting images, but attempting to exert precision over them is difficult.
Use Case 2: Final Product
Trying to prevent Stable Diffusion from generating it’s own images is a bit tiring, so let’s just add more pumpkins and call this done. The final product got a little bit of a fall vibe to it, but let’s be honest, not much. I didn’t document it, but I went on a long tangent attempting to turn the background sky dark / night time, but without any success. I found it difficult to truly give this a good fall motif without losing the original art. Again, this was 2+ hours of experimentation for what in the end felt a bit like bad clip-art. This use case didn’t feel anywhere near as good as the first one, though it did seem promising at first.
Case Study 3: Learn from our mistakes, try again
If Stable Diffusion is so good at generating completely new images, perhaps we shouldn’t try to start with a base creative, however let’s see what we can generate completely new? Let’s retry the two original case studies but from that angle.
Case Study 1: Input “clash of clans advertisement crowd of charging guys running towards a clash of clans giant” total time ~10 minutes, minimal tweaking settings
What a huge change, allowing the Text to Image to manage itself allows for some very unique and interesting scenes, and allows me to do a bit more “gambling” simply allowing the random generation of images to do the heavy lifting to find a good image.
Getting back to our themes, lets make a few themed images. Here’s two groups of four images for both Halloween and Christmas. It was so quick, these took a few minutes each and I didn’t even play with settings to fix anything up on them.
In total I spent mere minutes crafting the above sets of images, a fraction of the time as others, and arguably better looking scenes came out of it. To me, it felt like the less precise control I tried to exert, the better the results.
Final Thoughts & What’s Next for Marketers
Image generation needs more modular components for artists and marketers alike to manipulate. For example, outputting backgrounds, characters and objects as separate images and then composing them. This would allow flexibility later when working with the image. The landscape of image generation has been changing quickly and more and more the tools built around Stable Diffusion. I foresee further chaining together multiple AI algorithms and editing tools to enable more diverse use cases. I think what may come out of these are new specialized roles for artists who can create images from these tool sets.
Canva now has built in Stable Diffusion, though the results are much too constrained for this use case.
Marketers meanwhile will find new uses for newly created images that are not quite on brand. Think rapid concept art prototyping and much wider ranges of A/B testing. Think social media marketers able to respond to simple messages with witty replies that contain unique images tailored to that day’s news or memes.
I see Stable Diffusion as a boon, the clip-art of our generation bringing art closer to everyone and the tools for marketers to create better experience ads.
If you have any questions, feel free to contact me or reach out .
There is a growing discrepancy between the vision and reality of VR. For decades, science fiction and technology visionaries foresaw a future where consumers were able to freely traverse virtual reality like surfing the web. Virtual reality was seen as an open platform where millions of people could share, consume and explore content. Instead of this vision, we are building walled gardens without connectivity between different content. To open VR to its true potential requires innovation around linking content and open platforms. VR is inherently social, but first we must leap the walls tech oligarchs are constructing across the new virtual reality landscape.
Virtual Reality: The Vision
“The sky above the port was the color of television, tuned to a dead channel.”
Thus began 1984’s science fiction classic Neuromancer, which popularized the concept of cyberspace. Gibson’s virtual reality is notable because it was conceptualized before the web existed. This was explored in depth in early cyberpunk works such as Tron, Nueromancer, Ghost In the Shell, and Snow Crash which shared a future vision of the internet where three-dimensional metaspaces were linked and could be traveled at will. More recently this idea has resurfaced in the the popular, Ready Player One, which also contains visions of three dimensional metaspaces, though they technically exist within one game. Each of these works imagines society taking advantage of a networked cyberspace and the benefits of three dimensional spatial semantics.
The visionaries of science fiction foresaw virtual reality like the web in that content from these worlds is linkable through addresses. Thus anyone with a virtual reality device may visit any address and experience its content, as well as click on links to other virtual reality spaces. For example, a quiet cyberspace cafe, might contain links to coffee bean information, virtual chatrooms, ecommerce stores, bulletin boards or news content. Some of this might be hosted by the cafe, some might be links to other virtual reality spaces or web content.
In 2016 the VR platforms are controlled by tech oligarchs. Oculus Rift, HTC Vive, PlaystationVR, GearVR and even Google Cardboard all focus on building walled gardens. Current VR requires you to purchase devices to experience the content created for that specific VR device, similar to XBox and Playstation consoles. An extreme example is that Samsung GearVR doesn’t have native YouTube support as they see this as competition to their own content. These VR companies are backed by billionaires and technology giants, why would they mess this up?
Explanation via Analogy, 1994: The Information Superhighway
“This emphasis on profit has some in congress worried that the coming information superhighway may be dominated by a handful of big boys.”
Before the Internet was the popular form of the web we use today, there were competing models for how to network devices to deliver content. One famous example is the Information Superhighway which would connect ecommerce, videos, business, music to users through their living room televisions. Again, this idea was slightly before the web, and at the time was a competing model for what the Internet could be.
Brian McCullen, @brianmcc, said in his Internet History Podcast, that Bill Gates “along with his buddies in cable and telecom” thought they could bring the Information Superhighway to users “in a prepackaged and pre-designed way.” Yet, Microsoft and cable networks’ walled gardens “didn’t anticipate that everything the Information Superhighway hoped to be, would bubble up from somewhere else.” The world wide web exploded onto the scene with the first major web browser, Netscape, which opened up the idea of the web and an Internet controlled by no one.
The idea of cable networks controlling the access to the networks through television sets crumbled with Netscape’s free browser that could be downloaded onto any computer, for free, and then access any content across a growing web of mostly free content. Netscape also pioneered technology that made websites more accessible, safer and visually appealing. Their contributions to the web were open source and free for websites and competitors alike to adopt. As Netscape jumpstarted the dotcom boom that would define the 1990s, the concept of the walled gardens of the information superhighway was mostly forgotten.
2016: Bring Down the Walls
1. Linking
The idea of a shared virtual spaces has already been present in many popular games. Second Life, 2003, allowed users to create worlds and experience them together on the internet. Minecraft, 2010 alpha, user’s created and shared worlds was also the core mechanic. Still, these games are walled gardens. A user in Minecraft use a link to take him into a world in Second Life or anywhere else.
Links will give the ability to link any object in a metaspace to another metaspace or web page. Hyperlinks are the simple idea behind the web that makes it open and free. Anyone can link to any other page, and it is expected to be able to quickly load on your browser. iPhones and Androids were around for years before DeepLinking was created, this time we need to begin utilizing linking in VR content right away. Linking will force other content to be open and platform agnostic.
2. VR & the Cloud
This is an area that is ripe for innovations. High intensity graphic processing will need to be done in the cloud and streamed to VR browsers. Cloud Gaming has in development and deployment across a variety of platforms, yet very little of this has been open sourced. Google and Apple have also both taken steps to run apps on their servers and stream the content to interested users before they download the app.
3. Open VR Protocols
This part is moving forward, though slowly. It is an area that needs innovations and companies to lead the way forward. Currently we have browser giant Mozilla is leading the way with MozVR and open WebVR protocols with competitors like startup JanusVR creating it’s own JanusVR Markup Language. In March 2016 Googlesimultaneously released open source Web iframes for VR as well as Android app support for VR View.
Conclusion
We are in the early days of VR. If we want it to exhibit the open culture of the web, we need to begin building that openness now. That way when VR does become a useful tool for society, it will benefit the largest number of people rather than the current of VR’s walled gardens.
Recently my mother came to visit me in Taipei. On the drive from the airport we talked about my startup, Bubbleye, and how we predict a user’s interests in mobile ads. Her response: “I hate ads, who wants to see ads?”
She’s right, no one wants to see intrusive popup ads. People hate advertising, especially digital. It is seen as both annoyingly intrusive and ethically unacceptable in its casual invasion of privacy. Yet despite all this advertisements are human inventions, labors of love and ingenuity.
“Doublethink means the power of holding two contradictory beliefs in one’s mind simultaneously, and accepting both of them.” George Orwell, 1984
For those of us working in the advertising we both despise advertising, yet enjoy our jobs. As creators of the ad world this doublethink is our reality. We love ads, we hate ads. I love my job, yet I also encourage my friends to use ad blockers to improve their browsing experience. In Orwell’s 1984, doublethink is a propaganda tool used to edit understanding of truth, yet I see it also as a sophisticated way of understanding the world. To accept two opposing views is to have both knowledge and faith in the forces of the industry to naturally stabilize between the two. Similar to faith that supply and demand’s equilibrium is a place we want to be, I trust that the ad industry will maximize both ad supported free content and enjoyable browsing experiences.
So, lets break down these two sides of love and hate for the ad industry and see what we think the future might hold.
Did you know that ‘apology’ used to mean not an admission of guilt but a defense of one’s position? Plato’s Apology is a retelling of Socrates’s self defense at his trial. As a way of tying together the love and hate of the ad world, I would like to deliver both the modern apology and older definition of an apology for advertising.
I Hate Ads: An Apology for Advertising
The picture below is a screen cap of the highly personalized invasive advertising of the future as portrayed in 2002’s Minority Report.
Ads which cause unsolicited intrusions into your life which produce negative experiences is the cardinal sin of the advertising industry. Louis D. Lo Praesterecently articulated this “ethical conundrum” of the ad world’s ability to inject advertising into every opportunity in the digital experience but only with the hope of a positive user experience and the reality of mostly negative experiences.
The second great sin of advertising is violation of privacy. Societies are still deciding where it is an isn’t appropriate for advertising to collect information, but this effort to understand big data behind advertising is obfuscated by the unseen world of data collection. In addition, it is difficult to decide where to draw the line about individual pieces of nearly meaningless data, which once collected and analyzed can show potentially sensitive information about an individual. Society still has difficulty fathoming what it means that corporations can know my name, friends, locations, habits, and interests.
I Love Ads: An Apology of Advertising
“The best minds of my generation are thinking about how to make people click ads. That sucks.”
–Jeff Hammerbacher, 2011
Hammerbacher’s infamous quote is where I would like to start my apology of advertising. That advertising generates wealth for small businesses and content creators is something that is well understood and currently being discussed across many digital mediums. Less popular are the ideas that advertising both spreads information and breeds innovation.
Seeing an ad spreads information. Given that information is of high value and true, this is an extremely positive thing. The information is being spread in a non natural way, funded by money. Marketing campaigns can help the smallest and biggest players accelerate their natural word of mouth growth. Thus more efficient advertisements are more efficiently accelerating economic growth. For example, Tesla’s ability to hype its products is not only accelerating their growth, but accelerating the mainstream acceptance of electric vehicles around the world. This of course can have runaway effects and has caused governments to regulate commercials targeted to children, or possibly addictive activities. Advertising’s power to spread of information is overall a positive effect as it is moderately regulated.
Advertising breeds innovation because that is where the money is ready to be spent. While that may seem materialistic, some ad product concepts are expensive and difficult to develop. Capital support is necessary for research and development, which will later spread to other sectors. For example, big data research has been at the forefront of adtech companies. These innovations benefit machine learning and data mining in other industries which are not directly related to advertising yet benefit from this boon of advances.
Conclusion: Accepting Evolution
The evolution of the advertising industry is not lockstep with the society’s understanding of the advertising industry. Advertising has evolved so fast that society is currently still grappling with the wider implications of these changes. It is important for both insiders and outsiders to understand the forces moving advertising, and weigh the benefits and costs. This evolution of the advertising industry can be influenced by those outside of it, as it is currently by the uptake in ad blocking technology. This is a positive force that should be accepted. At the same time ad industry insiders are evolving the industry to show less but more targeted advertising. Balanced together these forces push advertising, innovation and society to better places.
