Saturday, June 15

WWDC 2018 Diary Of An iOS Developer

WWDC 2018 Diary Of An iOS Developer

WWDC 2018 Diary Of An iOS Developer

Lou Franco


The traditional boundaries of summer in the US are Memorial and Labor Day, but iOS developers mark the summer by WWDC and the iPhone release. Even though the weather is cool and rainy this week in NYC, I’m in a summer mood and looking forward to the renewal that summer and WWDC promise.

Photo in front of the San Jose McEnery Convention Center
WWDC (Image source) (Large preview)

It’s the morning of June 4th, and I’m reviewing my notes from WWDC 2017. Last year, I wrote that ARKit and Core ML were two of the big highlights. It was refreshing to see Apple focus on Machine Learning (ML), but there wasn’t much follow up in the rest of 2017. ARKit has spawned some interest, but no killer app (perhaps Pokemon Go, but that was popular before ARKit). Apple did not add to its initial library of Core ML downloadable models after the Core ML announcement.

Apple did release Turi Create and Lobe released a new interesting Core ML model maker last month. In the Apple/ML space, Swift creator, Chris Lattner, is taking a different approach with Swift for TensorFlow. But from the outside, Core ML seems mostly to have one obvious use: image classification. There doesn’t seem to be a lot of energy around exploring wildly different applications (even though we all know the ML is at the core of self-driving cars and whiz-bang demos like Google Duplex).

Another way Apple uses ML is in Siri, and earlier this year, I wrote about SiriKit and mentioned its perceived and real deficiencies when compared to Alexa and Google. One issue I explored was how Siri’s emphasis on pre-defined intents limits its range but hasn’t produced the promised accuracy that you might get from a bounded focus.

The introduction of HomePod last year only highlighted Siri’s woes, and a widely reported customer satisfaction survey showed 98% satisfaction with iPhone X but only a 20% satisfaction with Siri.

With all of this in the back of my mind, I personally was hoping to hear that Apple was going to make some major improvements in AR, ML, and Siri. Specifically, as an iOS developer, I wanted to see many more Core ML models, spanning more than just image classification and more help in making models. For Siri, I wanted to see many more intents and possibly some indication that intents would be a thing that would be added year round. It was a long-shot, but for AR, the next step is a device. But in the meantime, I hoped for increased spatial accuracy.

Finally, I love Xcode Playgrounds and iPad Playground books, but they need to to be a lot faster and stable, so I was hoping for something there too.

On the morning of WWDC, I tweeted this:

An image of a tweet with my WWDC wishes
My tweet describing what I was hoping for from WWDC 2018 (Large preview)

This wasn’t a prediction. It’s just a list of things I wanted to use in 2017 but found them underpowered or too hard for me to get started with, and was hoping Apple would make some improvements.

My plan for the day is to watch the keynote live, and then to watch the Platforms State of the Union. Those give a good overview of what to concentrate on for the rest of the week.

End Of Day 1: The Keynote And Platforms State Of The Union

The first day of WWDC is the keynote, which is meant for public consumption, and the Platforms State of the Union, which is an overview of the entire event, with some details for developers so that they can choose which sessions to attend.

Summary Of Notable, Non-iOS Developer Announcements

WWDC is not entirely about iOS development, so here’s a quick list of other things that happened to the other platforms or that are not very developer focused.

  • To get it out of the way, there were no hardware announcements at all. No previews and no updates on the Mac Pro. We’ll have to wait for the iPhone and follow-on events in the fall.
  • iOS 12 has a new Shortcuts app that seems to be the result of their acquisition of Workflow. It’s a way to “script” a series of steps via drag and drop. You can also assign the shortcut to a Siri keyword, which I’ll be covering below.
  • iOS will automatically group notifications that are from the same app and let you act on them as a group.
  • Animojis can now mimic you sticking out your tongue, and the new Memojis are highly configurable human faces that you can customize to look like yourself.
  • FaceTime supports group video chat of up to 32 people.
  • There is a new Screen Time app that gives you reports on your phone and app usage (to help you control yourself and be less distracted). It is also the basis of new parental controls.
  • Apple TV got a small update: Support for Dolby Atmos and new screen savers taken from the International Space Station.

Photo of the new Apple TV Space Screen Saver
Apple TV Space Screen Saver (Large preview)
  • The Watch got a competition mode for challenging others to workout-related challenges. It will also try to auto-detect the beginning and end of workouts in case you forget to start or stop them, and it now has Hiking and Yoga workouts.
  • The Watch also has a new Walkie-Talkie mode that you can enable for trusted contacts.
  • There are more audio SDKs that are native on the Watch, and Apple’s Podcasts app is now available. I expect third-party podcast apps will take advantage of these new SDKs as well.
  • The Mac got the anchor spot of the event (which is hopefully an indication of renewed attention). It will be called macOS Mojave and features a dark mode.
  • There are big updates to the Mac App Store, but notably, it now gets the same visual and content treatment the iOS App Store got last year. There are enough changes to the sandbox that Panic has decided to move Transit back there.
  • Quick Look in the Finder now has some simple actions you can do to the file (e.g. rotating an image) and is customizable via Automator.
  • Mojave will be the last version of macOS to support 32-bit apps and frameworks, which means the Quick Time Framework going away. It has seemingly been replaced with some video capture features in the OS itself.
  • Apple announced that they are internally using a port of UIKit to make Mac apps and showed ports of Stocks, News, Home, and Voice Memos. The new framework will be released in 2019.

The iOS Developer Announcements I’m Most Excited About

iOS Developers got some good news as well. They hit on the four major areas I wanted to see improvement on:

  • SiriKit now has custom intents, which opens up the possibilities quite a bit.
  • Create ML is a new way to use Xcode Playgrounds to train models via transfer learning, which lets you augment existing models with your own training data.
  • Xcode playgrounds now allow you to add code to the bottom of a page and run it without restarting. It’s hard to know if Playgrounds will be more stable until we get a real release in September, but this will make trying code much faster.
  • ARKit 2 was announced along with a new Augmented Reality file format called USDZ, which is open and was developed with Adobe and Pixar. Adobe announced some tooling support already. It will allow users and developers to store and share AR assets and experiences. In addition, ARKit 2 allows multiple devices to be in the same AR environment and supports 3D object detection.

We didn’t get an AR device, but it sure feels like we’ll get one soon. And it needs to come from Apple (not third parties) because running ARKit requires an iOS device.

Setting Up Your Machine

Everything you need is available now in the developer portal. To use the code in the article, you need the Xcode 10 Beta. I would not recommend using iOS 12 Betas yet, but if you really want to, go to the portal on your device and download the iOS 12 Beta Configuration Profile.

The only major thing you need a device with the beta for is ARKit 2. Everything else should run well enough in Xcode 10’s simulator. As of the first beta, Siri Shortcut support in the simulator is limited, but there is enough there to think that that will be fixed in future releases.

End Of Day 2: Playing With Siri Custom Intents

Last year, I wrote how you needed to fit within one of Apple’s pre-defined intents in order to use SiriKit in your app. This mechanism was introduced in 2016 and added to in 2017 and even between WWDC events. But it was clear that Amazon’s approach of custom intents was superior for getting voice control into more diverse apps, and Apple added that to SiriKit last week.

To be clear, this is a first implementation, so it’s not as extensive as Alexa Skills just yet, but it opens up Siri’s possibilities quite a bit. As I discussed in the previous article, the main limitation of custom intents is that the developer needs to do all of the language translation. SiriKit gets around this a little by asking the user to provide the phrase that they’d like to use, but there is still more translation needed for custom intents than for predefined intents.

And they built in on the same foundation as the predefined intents, so everything I covered still applies. In fact, I will show you how to add a new custom intent to List-o-Mat, the app I wrote for the original SiriKit article.

(Free) Siri Shortcut Support If You Already Support Spotlight

If you use NSUserActivity to indicate things in your app that your user can initiate via handoff or search, then it’s trivial to make them available to Siri as well.

All you need to do is add the following line to your activity object:

activity.isEligibleForPrediction = true

This will only work for Spotlight-enabled activities (where isEligibleForSearch is true).

Now, when users do this activity, it is considered donated for use in Siri. Siri will recommend very commonly done activities or users can find them in the Shortcuts app. In either case, the user will be able to assign their own spoken phrase in order to start it. Your support for starting the activity via Spotlight is sufficient to support it being started via a shortcut.

In List-o-Mat, we could make the individual lists available to Spotlight and Siri by constructing activity objects and assigning them to the ListViewController. Users could open them via Siri with their own phrase.

It’s redundant in our case because we had a pre-defined intent for opening a list, but most apps are not so lucky and now have this simple mechanism. So, if your app has activities that aren’t supported by Siri’s pre-defined intents (e.g. playing a podcast), you can just make them eligible for prediction and not worry about custom intents.

Configuring SiriKit To Use Custom Intents

If you do need to use a custom intent, then SiriKit needs to be added to your app, which requires a bit of configuration.

All of the steps for configuring SiriKit for custom intents are the same as for predefined intents, which is covered in detail in my SiriKit article here on Smashing. To summarize:

  1. You are adding an extension, so you need a new App ID, and provisioning profile and your app’s entitlements needs have Siri added.
  2. You probably need an App Group (it’s how the extension and app communicate).
  3. You’ll need an Intents Extension in your project
  4. There are Siri specific .plist keys and project entitlements you need to update.

All of the details can be found in my SiriKit article, so I’ll just cover what you need to support a custom intent in List-o-Mat.


0 0 votes
Article Rating
Notify of

Inline Feedbacks
View all comments
Would love your thoughts, please comment.x