108

Windows Phone developer hooks up Cortana to control his lights

Cortana Lights

We’re very excited to see what developers can do by integrating their apps with Cortana. It’s a very promising future for those on Windows Phone 8.1. One developer has already started to play around with Cortana. Matt Cavanagh, a guy from South Africa who is well-known in the Windows Phone developer scene, has hooked up Cortana to control his lights. Check the video after the break.

It’s a quick and early look at the possible integration of Cortana and your favorite apps. Matt used a Cortana on Windows Phone, a Netduino, a Bluetooth module and a normal 240v desk lamp to get this early concept up and running. You don’t even have to work too hard to see Microsoft getting into home automation and have Cortana truly be your personal assistant by controlling everything around you.

Pretty cool concept from Matt and we're excited to see him take it a bit further. In the meantime check out VALA: Alpha, his new Windows Phone 8 game 

Any other developers out there have fun concepts with Cortana running? Be sure to share with us!

Via: @roguecode

22
loading...
449
loading...
124
loading...
0
loading...

Comments

There are 108 comments. Sign in to comment

kidjenius says:

Shut up and take my money

Aashish13 says:

Fuck how did he do that. How hr connected it??

paulheu says:

App controlling lights over Bluetooth.. There's a lot of these projects on the net and it's fairly easy to program.

RogueCode says:

Yup. I did some stuff that Microsoft's IOT "traffic light" demo at BUILD was based on. Checkout the source and tutorial here: http://blog.roguecode.co.za/Internet-of-Things-Azure-cloud-Netduino-othe....

Read the article butt nugget.

kenzibit says:

Why does she keep on calling us Chief??

KrightonX says:

You should play more video games or read up on "who" Cortana really is.

kenzibit says:

Dude I know Cortana more than you do...maybe before u were born but at least we are not Master Chief. Shouldn't she call us by our names?

the guy has set his name as Chief.

tixem says:

Nope, i gave the name "David" yet it calls me chief no matter what. Kinda silly, why should I give it a name if zero fucks given? 

kenzibit says:

This is exactly what I'm trying to explain here. I've set my name, but she never calls me by it. And when I ask some questions she replies with chief before answering. She should at least call my name. Trying to explain this and people don't get it.

ngocthua92 says:

You call your boss by his name?

Hahahahahahahhahahahahaha EPIC!!! That was so funny but I think maybe she can't really pronounce those names so they automatically make her say chief.... Maybe

hahahaha i have to agree that was funny as hell.. it weired though cause she asks you how your name is pronounced and stuff in the initial setup.. but never says it.. asking her what is my name? who am I? etc. doesn;t work either.. I tought her to say Kakarot (DBZ) adn she said it perfectly but I've never heard her that that since i set it up.. which is pretty sad.. 

jibreezy1982 says:

How hard is the name David to pronounce?

Wael Hasno says:

Just be glad you're having a female voice calling you chief..

WinFan1 says:

Cortana is the artificial intelligence of the master chief.

kenzibit says:

But we are not Master Chief. She should call me by my name right?

WinFan1 says:

Every one should strive to be like the master chief, he saved the universe you know.

jibreezy1982 says:

So did Shepard..

TechFreak1 says:

@kenzibit you can have Cortana call you whatever you want, you are asked when Cortana is first launched and can be edited in the settings via the tile or settings - application. Plus many prefer to have her call them "chief" :P.

jklier says:

I have my regular name set in Cortannas settings and all of the appropriate boxes checked and have not had her call me by name yet. Tried resetting the app as well with no luck.

Speak for yourself. JOHN 117 REPORTING FOR DUTY!

ಠ益ಠ

vkelkar says:

Guys relax its beta... It will evolve... Don't worry it wud be done.... For now enjoy 8.1 and cortana

chrispyogurt says:

Maybe its her habit. Bad habit (or good) dies hard.

Fizoto says:

From what I see, it's "chief" not "Chief". Probably meant in a generic way.

saket87 says:

Goto apple store and buy Philips light with changing colors and get ONI light control app, it works with Cortana

DennisvdG says:

You lost me at "go to apple store"

Lol

Flagz says:

There's a store that just sells apples? Hope it at least sells a variety of Fresh Apples, but seriously there's so much more fruit out there. Why get stick with an apple.

HotJava says:

U can buy Philips lights on amazon, no need to go to the enemy's camp :)

greedypnguin says:

Philips hues are a tad over priced tbh

saket87 says:

I got it at 200, it's over priced but worth it I guess, I like it though

wpgeek820 says:

That's cool. So developers can program Cortana's speech capabilities? Awesome.

CJ Thunder says:

Well they could in 8 too, but this is cooler just because.

sirfergy says:

For launching apps, you were bound to specific phrases, but with Cortana you can now pass in random grammars enabling much richer voice apps.

Yangstax says:

MS provides developers Cortana API which can directly access to the databases and applications.  The results of voice query can be programmed to send command to external devices via BT.  For this case, a BT supported electrical switch is used to turn on and off the light.  In theory, all kinds of control can be initiated by Cortana.  Neither Apple or Google provides API to Siri/Google Now developers.  So this is a unique strengh of Cortana.

Now, thats Awesome!!

Could you also please let me know other functionaliteis of Cortana?

 

Thank you,

Chaitanya

chaitanya.hps@live.in

Yangstax says:

Once the Cortana API is on the developers' hand, the creativity starts.

ymcpa says:

that is one major advantage over siri and google reviewers have been stating.

bc3tech says:

It's worth noting that the same speech capabilities have been around since WP8 launched. Just in TellMe form and not Cortana form. So... yeah...

cool idea though.

ap3rus says:

Well not the same actually, in WP8 developer had to program all specific voice commands word by word manually, OS provided developers with just the feature of translating audio to the text and that's it. WP8.1 gives developer not only the speech recognition, but also a natural language processing - so that there're machine learning/AI algorithms that process the recognized text to sort of get the meaning of the used wordings. That simplifies things for developers like A LOT.

bc3tech says:

I'd be very interested in seeing the documentation you're referencing on TellMe vs Cortana integration from a developer standpoint, then. Having done the Code Challenge for Cortana implementation at BUILD, I saw no difference. And to-date have not seen any new documentation saying "for Cortana, do this!" - it's all still VCD (voice command definition) files from what I have found and implemented on 8.0 and 8.1

ap3rus says:

http://channel9.msdn.com/Events/Build/2014/KEY01

Head to 1:33:40, there's the mention of what I've described.

bc3tech says:

I was at the keynote. I want hard implementation details. Wp8.1 is out, the SDK is out, and i've seen no change or any new doco on how developers integrate w/ voice reco.

ap3rus says:

If you were at the BUILD why didn't you ask WP team about it in person?

Anyway I don't think WP8.1 is fully out yet, the documentation and SDK seem in preview at the moment, some subjects are not fully covered.

bc3tech says:

lol. yeah just walk to up belfiore and ask him. why didn't i think of that.

 

after some cursory Binging I've come across the following:

http://www.bing.com/blogs/site_blogs/b/search/archive/2014/04/14/integrating-store-apps-with-cortana-in-windows-phone-8-1.aspx&adlt=strict

which directs devs to look at the source code for the MSDN Voice Search application.

Indeed we still use the VCD files, but can put special keys in to the file to get different results.
The salient point here is this is only valid in-app, and system-level commands still require knowing the phrase ahead of time, based on how I read that page anyway.

Also given this, it's interesting to note this is the only doco i know of; this app being released and open-sourced. And what is being shown in the video on this post is *still* 110% possible *without* Cortana.

leoniDAM says:

You are right, the difference in the Speech APIs between WP8 and WP8.1 are minimal you still create your VCD, and you can use custom SGRS grammar for speech recognition, like on WP8. Here there is a Demo showed at the Build: http://channel9.msdn.com/Events/Build/2014/9-2663 Here there is the MSDN documentation: http://msdn.microsoft.com/en-us/library/windowsphone/develop/jj206958(v=vs.105).aspx And as you can see are essentially the same.

So for now the real benefit of Cortana, for the developer is the speech engine that interprets the voice.

But for the future the idea is to expose more capabilities to the third-party developers.

leoniDAM says:

But you are wrong on one thing:

To compare TellMe and Cortana.

Are two different thing.

Cortana substantially is an interface but under the hood there are many component on the cloud and on the device.

For example there is the engine that track the information, there is also an engine that recognize what the user says, there is Satori, ect.

The combination of all this part give as Cortana.

But TellMe was only the engine that recognize hat the user says, so in some way is still in Cortana and is par of Cortana. This speech engine is what intercept a speech and translates it into a bunch of 0 and 1 which can be converted into a text string.

But not only you have to translate a speech in a texst string, but also to perform the right action with that string.

Moreover a digital personal assistant is not only about speech, is also about to know an find the information that the user want and to inform the user about this.

Remind to wish happy birthday to a frient when he call you or to buy the milk when you are near a market, don't necessarily involve a tecnology like TellMe (in fact you can make a reminder also with keyboard), but a speech engine is involved when you use your voice to create this reminder, to inform the personal assistant that you want to be allerted about this.

TellMe was an engine that provided the speech capabilities in some Microsoft product, in the 2011 the team of TellMe (TellMe originally was a company that provide speech engine and software base on speech tecnology acquired by Microsoft in the 2007) was moved under the Bing Division and, starting from that point, Microsoft no more refer to this speech engine like TellMe but with a more genereic Bing Speech and it is not a real product that you can install or something similar, but is more like a tecnology, a distribuited tecnology (not only in the sense of in the cloud but also in the sense of a many interdisciplinary technologies that work together).

So is wrong to compare Cortana vs TellMe because "TellMe" is part of Cortana and that can take advantage of TellMe tecnologies

This was the first thing I told my friend, when Belfiore announced that Cortana can be used in apps. I was gonna do this, dammit!

LSDigital says:

Now you can do using wifi or the internet :p

so I think e can update Cortana w/o global update?(like a music, video, game hubs?

teoami says:

Cortana is essentially updated "on the fly" all the time since most of what you ask her to do is processed in the cloud and not on your phone itself.

leoniDAM says:

Some part is processed on the cloud some other on the phone

Like a Xbox music"some part in phone, some in the cloud ☁

Mohit Jain4 says:

Nicely done matt

peachy001 says:

As soon as I saw the headline, I thought that it sounded like something that South African Llama wrangler would make. Genius. So jealous.

peachy001 says:

I console myself with the delusion that the only thing that separates our achievements in programming is my lack of spare time.... It isn't that, turns out you are more able too! Ha ha. Kudos to you and thanks for responding previously with regards to the Bluetooth remote control vehicle. Don't stop.

RogueCode says:

It helps that I do this stuff for a living too ;-)

peachy001 says:

Advantage Rogue Code. Keep it up. Hopefully we may see an Xbox One game at some point.

DVELOPinc says:

Been waiting to see what your next project would be after you had posted this one! My fav is still the car for going under your house, lol.

guyjkh says:

Well done !!! Awesome!

KrightonX says:

Cortana, when I do this, <redacted>  I need you to make this face :O

SamiSaifDine says:

It's like "show me the news" then the WPCentral open up!

Mzantsi fo sho fly that flag!!!

Wow, now that is amazing. Hook this up to the xbox one and have your home entertainment controlled wherever you are in the house

superpos says:

It's nice!
I'd like MS could implement a voice command to activate Cortana instead of pressing the button, something  like "Calling Cortana....>>> request...." and 'til the end with something like "end calling Cortana", to stop it.

I understand probably the OS should always check for a command coming in the mic and maybe it could drain the battery, but, well, I'm not a technician so I don't know if it's really doable, but I'd like a voice command to activate it.
 

MDboyz says:

So people can cry about 'always on' like the XboxOne kinect? ...

But it would be nice if they give you option to have Cortana always on too ...

MrVol84 says:

How much more awesome would this be if he could do it hands free?

Aashish13 says:

Hope Cortana in the future can give us cricket updates here in India.Satya might love it when he plays cricket

mccasive says:

Yeah, she already giving me basketball, baseball and soccer updates here.

Aashish13 says:

Not the cricket updates but when I ask her. In India cricket apps and info are much needed.

Monique Mark says:

But they said very clearly that cortana is only American for now. Obviously Americans don't care about cricket. You went around ms to get her on your phone in India what did you think you should get. I'm sure when she comes to India you could get ask the cricket scores you want. Currently Google now does not recognise "did west indies win" for me. Lol

saket87 says:

It works with oni light control too

abond32 says:

Good now make some apps.

sip1995 says:

Cortana can do more than siri and srewgle now.

LaNiQuE says:

This is pretty damn cool I love our developers

joltcoder says:

Snap Switch has had this before Cortana. Now it's even cooler. Works for all of my insteon and x10 devices.

Microsoft Agent was command based, but it had the capability years ago. I had "Merlin" turning on my lights, giving me the time.... and small preprogrammed conversations.

No where near as cool as Cortana, but MS has had the speech capabilities to do this since 2000.

A small trick with cortana. To get ur normal Bing search screen with wallpaper and Bing VISION immediately while using cortana, switch off the location and click the search key. Default Bing search will appear. Cortana and Bing search with vision can be toggled by using location on/off. May be someone else might have already covered this, but I just wanted to say :)

Yangstax says:

There is a 'QR for Cortana' app you can install from the Store.  Tap on that Tile to reach Bing Vision.

mrdvos says:

Cortana activated using voice would be cool .

Aashish13 says:

How it should be commanded then? 'Cortana'

lemonsteveo says:

Unfortunately, even if MS adds this, it wont be available on existing phones. You'll need a specific chip to handle the always listening part like the Moto X. Oh well maybe a future Lumia.

InlineV says:

And he didn't have to hack it using a man in the middle vulnerability like some guys did with Siri.

Pradyun says:

Great creativity

immy79 says:

Or, just get a Wemo and use IFTTT to have Cortana send a text to turn lights on or off. This seems overworked to me.

Didn't they already have something like this with Oni for Phillips Hue? http://m.wpcentral.com/oni-light-control-philips-hue-support-windows-phone

Reebs Reebs says:

It's a shame that Nest Thremostat does't develop an integrated app with Cortana like that! I would love to have it. But again, they don't even have a Windows Phone app to start with! If you have Nest you can vote for the WP app request here. https://community.nest.com/ideas/1014

DVELOPinc says:

Good luck with that. You can't even get Google to give us an official YouTube app much less a Nest app.

Skippy07 says:

Can't wait to use Cortana to control my flying suit of armor!

tavernlong says:

The possibilities if added with CastleOS which already supports Kinect.

https://www.castleos.com

 

Grasshopper already takes control commands through Cortana to control your entire z-wave home network. But I wish the commands were simpler and more adaptable. The Grasshopper dev needs to make these commands simpler.

rajantiwari says:

Now this what makes iOS and Android fellas envious about us.

Zuka_WPC says:

When does Cortana say your name out loud? I've seen it onscreen but never heard her say it out loud

jasongin says:

You can do this and much more with the Snap Switch app for WP8. It understands lots of voice commands and works with several off-the-shelf home automation controllers: Hue, Vera (Z-Wave), and ISY (Insteon).

I am the developer of Snap Switch. Apparently I should have submitted a video to WPCentral.