164

Lumia 1520 beats iPhone 5s in Memory benchmark test, but just misses on the overall win

Lumia versus Lumia and Windows Phone against the iPhone in our benchmark matchup

Benchmarks don’t mean much to the average consumer, but they’re a fun way for us to compare and stack different handsets.

Earlier today Basemark OS II launched on Windows Phone, a benchmarking application for Windows Phone. We’ve had a few other benchmark apps in the Windows Phone Store before, but this one also has apps on Android and iOS. We thought it would be fun to run the test on all the Windows Phone devices we have in our possession, in addition to an Apple iPhone 5s for comparison.

How does a Lumia 1020 fair up against a Lumia 925? Does that extra bit of RAM in the new Lumia 525 really matter against the Lumia 520? And how fast is a Lumia 1520? Read on to find out!

Lumia 520 and Lumia 525 Benchmark

How does an extra 512 MB of RAM matter for benchmarks? We put the 512 MB Lumia 520 with its 1 GHz CPU up against the updated Lumia 525 and its 1 GB of RAM (same CPU) to find out.

As expected, that extra bit of RAM improves performance, especially when it comes to running graphics. That shouldn't be a surprise for those wanting to run games on their Lumia 520, as clearly the lack of RAM hurts it in gaming. Now you can see why new games aren't optimized for 512 MB devices, as that reduction in RAM is a bottleneck. Still, the difference is modest, meaning the everyday experience shouldn't be too drastic.

Lumia 520 and Lumia 525 Bench

Lumia 520 and Lumia 525 Camera test

Lumia 520 and Lumia 525 Camera test

Lumia 1320 and Samsung ATIV S Benchmark

On paper, the giant six-inch Lumia 1320 and the more modest ATIV S seem very different, but in the benchmarks, the Lumia 1320 is only a hair faster with improved 'System' performance. That's most likely due to its CPU, which is clocked slightly higher (1.7 GHz versus 1.5 GHz on the ATIV S).

Lumia 1320 and ATIV S Bench

Lumia 1320 and Samsung ATIV S Camera test

Lumia 1320 and ATIV S Camera test

Lumia 925 and Lumia 1020 Benchmark

Once again, the Lumia 925 and Lumia 1020 are very similar in specs (putting aside that whopping camera found on the Lumia 1020). But one area of difference? The Lumia 1020 has 1 GB more of RAM.  As a result, it narrowly beats out the Lumia 925, though the lead is modest, borderline negligible, to say the least.

Lumia 925 and Lumia 1020 Bench

Lumia 925 and Lumia 1020 Camera test

In case you needed proof that the massive Lumia 1020 camera does slow things down when compared to the Lumia 925, here you go.

Lumia 925 and Lumia 1020 Camera

iPhone 5s and Lumia 1520 Benchmark

Apples, meet oranges. While the Lumia 1520 and Apple iPhone 5s are altogether different for hardware and their OS, how that hardware performs is of concern here. The 5s famously sports a custom,  64-bit A7 processor, which is up against the Lumia 1520's cutting edge quad-core Snapdragon 800, clocked at 2.2 GHz. No surprise then to see Apple handily crush the Lumia 1520 in terms of raw System performance.

Interestingly, the Lumia 1520 bests the iPhone 5s in the Memory department, with 3x the score. As a result, the Overall scores are similar between the two flagship phones, but you have to give the ultimate win to Apple, who narrowly edged out the Lumia 1520 (999 versus 897).

iPhone 5s and Lumia 1520 Benchmark

iPphone 5s and Lumia 1520 Camera test

The Lumia 1520 certainly benefits from the Snapdragon 800 with a much improved camera over the Lumia 1020--at least in terms of raw speed. It can't catch the super nimble 5s though with its 97 rank, putting it well ahead of the Lumia 1520. But hey, 20 MP versus 8 MP matters too.

iPhone 5s vs Lumia 1520 Camera test

Wrapping it up

What do we learn from these benchmark tests? More RAM becomes an issue for Windows Phone when it's below 1 GB. While 2 GB also helps, its gains over 1 GB devices are modest. We also can see just how much better a quad-core Snapdragon 800, clocked at 2.2 GHz can help the OS become faster and faster--even for the camera. Clearly, the Lumia 1020 could have benefited from this newer chipset. There's always next time, Nokia!

Say what you will about Apple, but they do hardware right, going so far as to have their very own silicon for their iPhones. It pays off for system speed, but it looks like their memory architecture could be optimized a bit. At the end of the day though, it performs nearly on par with the Lumia 1520.

Finally, we need to keep in mind that a phone may score higher or lower than another, but at the end of the day that’s no indication of how it’ll behave in real-world use. Windows Phone is clearly a mobile-optimized OS and it will only get better as newer hardware options come online.

Editor's note: although all apps were closed when running the benchmarks, due to various uncontrollable factors on each device, we expect there to be some variation between your results and ours. But the overall trend should be spot on.

Daniel Rubino contributed to this report

5
loading...
0
loading...
76
loading...
0
loading...

Reader comments

Lumia 1520 beats iPhone 5s in Memory benchmark test, but just misses on the overall win

164 Comments

I'm not sure about that. It's really nice hardware, just too small for me and I think the OS design is a bit wonky. Hardware though is damn impressive, even the camera.

Hardware is excellent, don't like the OS. But their camera speed is impressive. Wish Nokia would lock focus even after you snap a picture and continously auto focus prior to pressing the shutter.

Well, is up to individual preferences when comes to OSes. I have friends not liking WP live tiles and metro too. And true to the auto focusing of Nokia camera. I have hard time focusing on foreground objects when in dim environments.

Agree with you about the screen, i remember the days of making fun of big phones, whereas now, it's bigger is better (upcoming 1520 owner)

TBH, they usually get an extra 6 months, at least, compared to competition. While WP got support for S800 after 5S released, it's been out for quite a while.

Agree with you 100%,i am the owner of an Iphone 5 and a lumia 820, and i really prefer the Iphone,so if apple comes out with a bigger screen,the lumia 820 will leave immediatly,there is especialy one thing that i hate of the Lumia,it's the price going down so fast,i've had it for 10 months and it's had a price fall of 40% since i got it,the Iphone has droped 15% and only because the 5S came out otherwise it would have kept better value.(price drops are common to samsung too)so if you want value for your money,get an Apple !!

Just my opinion,

regards to all !!

Uhm... Bigger Iphone is coming in March... 5.5" I believe.  But in September... the Iphone Air with a 4.7" perfect screen.... that's the game changer right there.

I've been trying to get my hands on the Lumia 1520 before then but I already know... my love affair with Windows and Android are numbered.  

Perhaps but being as though I was hanging in New Orleans over the holidays with both my 1520 and 920 taking pictures just as the sunset and into night.. (Though I only used the 1520 since I just got it). My friend put his iPhone away and sad you just take the pictures. Of course he still took pictures later. But no comparison between the two. Although his didn't look awful. Same results yesterday while at the park with another friend who loves taking pictures with his 5s. It is waaay too small though.

Wish they'd stacked the S4 and the HTC One, too. S4 because it's overrated, and the One because it's just awesome. Love my 925, though =)

Its the only iOS  device so if you like iOS better than WP and Android, which a lot of people do, then it is the best device out there.

To be exact, it's the screen resolution that should matter in that case. The screen size should be irrelevant. The iPhone 5S has 1136 x 640 pixels vs Lumia 1520 at 1920 x 1080. Yeah, a lot more pixels to fill. :)

It's awesome but the screen is too small for me. The 5 is amazing too. I miss it a little but wouldn't go back to it, I hope. I have a 920 now!

Everyone ready to bash competition. I love Windows and Windows phones, and they are my chosen OSs, but Apple is pretty damn good hardware-wise. That's more objective than subjective I think.

Yep the problems with iphones have always remained the same: fragility and price (i'm not counting software here). Oh and the screen is waaaaaaaaaay too small, although that is subjective.

Apple makes great hardware, no doubt about it. Sometimes people here don't give other tech companies enough credit. You may not like Apple, but they certainly offer some great products.

Also, when the Nokia deal is all finalised and MS can have involvements with Nokia hardware, hopefully they will work on further optimising the OS for the hardware.

Really like what you said here, unfortunatelly i did read an article about MS plans for hardwar after finalizing Nokia's deal.. i'm afraid they won't keep the great work and effor Nokia managed to do with Lumia hardware designes and color portoflio....

 

i also have to agree that Apple make a great hardware despite the screen size for their iphone. but when i hold my lumia 925 i totally feel in love with how it was shaped and designed.

We needed a great smartphone in a budget and Windows phones are proving. If Windows phone didnt exist I would have been stil using my small blackberry device. I started hating droid since its birth, I used to like iOS a lot, but loved Windows phone.

"We thought it would be fun to run the test on all the Windows Phone devices we have in our possession, including the iPhone 5s" ??

How else are we supposed to compare things? Or do you want your humble editor overlords to live in a Windows Phone bubble? I can point you to a few sites that are like that, lol.

(I also have an HTC One, though was too lazy to re-charge it and an iPad Mini Retina).

i think he may have been making a grammar critisism, if you said 'as well as the iPhone 5s' he may have not got confused.

Damn, if they managed to make the tests comparable between the two operating systems I don't know if I would call that a narrow edge.

These kind of things does not matter to me and i don't think many consumers care anymore. I love my 920 to death. I won't put this phone down in atleast 1.5 years...

 

May i add that the 920 is the first phone in my whole life that i think is perfect in every single aspect. I have NOTIHING to complain on. I will write Nokia(or Nokia team in MSFT) a love letter because of this phone. :)

I might add that i will write a love letter to the Nokia team, this phone has soul. It feels like a person, it feels like a life companion there to help you trough your day more than a thing to play on and check the time. I honestly can't let it out of my sight sometimes...

Sometimes i just find myself just stareing at the home screen becase i can't let go of how totaly amazing this phone is.

make them buy phones every year as zombies that leave them without updates or outdated, iPhone have secured three years and as the first day, I go back to iOS definitely the best option.

You sure know we WP users get updates?? iOS is def npt the best option... too small, dont like it.

Well... it's like a person to me that i love dearly. I did not marry it but i might aswell because i love it so much :)

The majority of consumers don't give a rats azz because they have no clue what it means. All they know is their buying a great phone and that everyone has it. To the minority tech sites and and techies, every ounce of juice matters. They will have an explosive diarrhea if there's a 0.1sec lag.

I have a 925 and miss my 920. Felt Awesome in the hand and had wireless charging built in. The way it is supposed to be

I am exactly on the same boat there. I have never felt more content with my cell phone until I own my current Nokia Lumia 920. In fact, I still am so content that all the new phones Nokia has launched so far have not gained my interest that much, not even the 1020 or 1520. 1020 is basically a similarly-speced 920 with a much better camera, and 1520 is just way too big.

I am still waiting for the true next iteration of Nokia Lumia 920, with better internals and camera, thinner and lighter body, and 4.5" - 5" screen size. Lumia 930, maybe? :)

The iPhone beats all the smartphones in touchscreen sensitivity and ecosystem..if Nokia figure out how to resolve these problems it would be the dream-phone .. (for now)

I think Nokia has done alot of work. MS is the problem. Remember ringtones and multitasking? The user community had to bitch for them! When we did get, their both half assed!

You might be right, but still they were working at 2 sides, everything wasn't clear enough and that's why they had to push updates to fix alot of issues which shouldn't be there in the 1st place.

i guess Lumia family will get better after MS takes full control.

ecosystem sure, but then again that is to be expected since they had a huge head start. Sensitivity? Not a chance. Touchscreen sensitivity is better on lumias since it even works with gloves.

By 'sensitivity' the OP might have meant response time. The iPhone does have a faster response time. I don't remember the link to the video that showed the results and demonstrated how a small time lag was actually noticeable. IIRC, the iPhone response time was about half of the Lumia.

Definitely. iOS is also very clever about how they make transitions and give the impression of making things happen fast. That's not a bad thing, it's very frustrating when NOTHING happens and you don't know if you clicked or not.

Well, I don't think I can agree with you that the iPhone beats all the smartphones in touch screen sensitivity?! How? Ecosystem, yes, overall system performance yes = same company made hardware and OS resulting in excellent performance and well integrated firmware but touch screen sensitivity, absolutely NO!

These benchmark tests were not all executed on the same days? I know this is off topic. But kinda weird that some camera tests were execute on different dates.

With that said, I can't believe the iPhone beat the 1520...

Same days, even same hour. Not all of our phones have live SIMs in them so time/date gets thrown off. I doubt that's a relevant factor in the scores, though.

Mine is about that:

Overall: 1029

System: 1195

Memory: 1390

Graphics: 1199

Web: 563

 

I wonder how consistant this benchmark is?

Tell me something. Is it difficult to walk around with such a massive phone. Does it struggle to fit in to your pocket? Would you rather have a smaller, more portable phone? Or is the huge screen completely worth it?

Its awkward at first but once a week has passed it feels normal. And once it feels normal its very difficult to go back to Lumia 920.

There is a period of adjustment, but after about a week or so you're able to use it without holding it awkwardly. Fits in all my pockets so far, and I wouldn't trade the screen for anything anymore! I can even type with just one hand.

I would have liked to see better scores for the 1020, but I wouldn't trade it in for any other phone currently available. As mentioned by the author, the speed of the hardware is but one consideration for overall value.

Mine scores are also better:

Overall: 496
System: 505
Memory: 855
Graphics: 350
Web: 397

 

btw. Has anyone noticed that even L520 has higher score than 5s in memory section? lol

Daniel would it not be much more usefull if the benchmark did take the different OS's into account, meaning how efficient they are at fully utilizing the hardware at hand. I'm not sure if that is feasible, but am just throwing the idea out there. After all it is pretty meaningless if one phone has a higher score than another but, due to lack of software optimisization, still runs slowers when performing one or more tasks.

I don't see how you can even do this unless the phones you're using has the exact same hardware as the other phone, but with different OS's.

the system score between wp8 and IOS says it all
as much as i love wp
yes its smooth, but its also slooooow
resuming the apps, opening the apps, they all take too damn long compared to ios
i was really hoping for speed improvements above "features" in wp8.1

Lumia 1520 opens apps faster than android (not sure about ios) and loads web pages faster than android and iPhone 5s, so WP is fast too

My lumia 920 managed to get better score than both lumia 925 and 1020. 512 overall, system 504, Memory 963!, graphics 353 and web 402

 

i see im,not alone/ sending from phone

I'm kinda confused as to the difference between phones with this benchmark.  I tested a 1520 and got a 1008.  I've also been forwarding it on to everyone I know and have this table of results so far:

 

 
Overall
System
Memory
Graphics
Web

HTC One
486
922
166
1173
311

iPhone 4s
246
402
110
118
698

LG G2
773
1276
277
2264
447

Nexus 5
957
1267
415
2304
694

Lumia 810
470
448
756
348
415

Lumia 1520
1008
1090
1391
1221
558

Galaxy S4
621
1269
214
1173
466

 

 

 

Could never understand this benchmark thingy. Ofc, S800 > S4, but what's the point of this pixel by pixel comparison (for the smartphones ofc). 

Numbers are always cool. But does it have any meaning? I remember the days benchmarks where made to favor specific hardware. Think crunch benchmarks, where some benchmark could run the test on some hardware in cache.

yep I have GDR3 and my 8x is 1 month old I got it unlocked from Amazon Germany for just 169 euro and ofc its limelight yellow :)

Mine is 1yo (unlocked too). I removed 1/2 of the stuff, games and images and Memory Increased to 694. May be I should try to reset it...

I absolutely love Windows Phone for what it is. But I'll have to admit it falls short in terms of features compared to other os. At least the hardware side of things has improved by a lot. The OS still needs to iron out its shortcomings and introduce more useful features.
It's been over a year since wp8 came out, but we need more changes beyond cortana, notification center. App isn't a problem for me. They are coming and wp isn't losing momentum anytime soon.
IOS and android changes at a rapid pace and Microsoft needs to speed up. Nokia has contributed more to the os than Microsoft did for the past year.
When it comes to iPhone, I'll give it a second look when it has a larger hardware and less cartoonish UI.

What "rapid pace" is the iphone evolving at? how long did it take for io7 to come out? and in the end it came out full of bugs and far from optimised. Android is the only one that now seems to be updated quite regularly, even if only a few devices get these updates.

I find this article rather amusing. The iPhone 5s is the most powerful iPhone to date, & Apple touts it as the best smartphone on the market, yet people don't realize that Apple is paying Nokia royalties for using their patents.....hmmm, good one, Apple.

I don't care a bit about the 5S in this. What SHOULD have been put to the test were Android devices, namely the One and S4. They run on the same chips as their Windows Phone rivals (save for the Exynos-powered S4 variants). I'd like to see how the Android devices compare to he Windows Phone ones when the hardware is on a level playing field.

Yes, the One runs on Snapdragon 600 but  GSM Arena lists the S4 as running Snapdragon 800 but Phone Arena has it with Snapdragon 600. Other phones that run on Snapdragon 800 are the LG G2 and the Note 3 (although some run on Exynos 5). But I think a benchmark comparison has already been done on the phones that run on this Qualcomm chipset but can't recall where I saw it. I remember distinctly the 1520 beat the Note 3, and G2 in that benchmark comparison.

Daniel before posting this piece I think you should have done at least 3 tests on each phone and get the average of the three, since its clear that results vary quite a bit, with many people getting scores with the 1520 higher than the iphone 5s.

So this confirms my theory that GDR3 is not optimized for the snapdragon 800. It's interesting to note that the Lumia 1520 scores 1000 System score while a GS4 with snapdragon 600 scores 1134 (on my friend's phone). Windows Phone is a highly optimized operating system and the fact that it's processing slower than lesser specced android phones is kind of dissapointing. 

my 1520 scored a 986 with the weather and facebook app running... you guys should probably retest several times and then average the scores.

Letting the phone sit aside for a couple of minutes and then repeating achieved 496, graphics 340, Memory 909.

The sleekness of iPhone 5S + the durability of Nokia + the speed of A7 + the awesomeness of WP8.1 = too good to be true sadly. :(

Just tried with "Multibench 2"and i got a better result with my Lumia 820 compared to the graphic board that they achieved with a Lumia 1020 ! Now it makes me think,is Nokia selling us brass for gold,or are these benchmark apps just phone fuckers ?

oh the tiny iphone 5s beats the lumia 1520, sure, it got a better system score but can you really notice why? ios vs windows phone... cmon we already know which one is heavier in terms of pure usefulness that translates to hardware requirements, graphics? oh really.. a tiny apple thing running at less than 720p resolution against 1080p 60fps, that it beats the lumia graphics by ~100 points is just disappointing, and we dont even need to talk about web if that test most likely rates for having the most HTML5 features enabled
 

Am I missing something? Tells me that these features are only available in the full version but I don't see another version or any payment methods...?

I don't know if I would trust this app.

I completed a benchmark on my 620, and got 320 as an overall score. Then I did a reboot, and ran the test again and only got 308. When I ran it again in airplane mode, I got 310.

Not inspiring a lot of confidence here.

I got much more varied results. On my Lumia 521 I got between 160 to 295 (overall)... quite the difference. The ranges within the individual categories varied a lot too. Definitely wouldn't trust this very much.

Is this benchmark relevant in any way? I mean the one between the Lumia and the iPhone. For it to be relevant you'd have to be using the same application as a reference, but we're talking about two different OSs - meaning two different apps.

Are the algorithms used identical? Written in native C++ for both platforms? Or how can the scores be compared in any way?

I don't think you can. I think that cross-platform these numbers represent nothing.

"Clearly, the Lumia 1020 could have benefited from this newer chipset. There's always next time, Nokia!"

That's just brutally cruel.

Apple is great but when it comes to real life situation, i cant chose it. Its over priced. Screen is bit small and i feel like its width is not enough. Apple have best internet browser but its scrolling sucks. Apple should really change that. I like the camera of iphone 5s but I LOVE LUMIA 1020 :). Its  alot slower but pictures are so beautiful on the 1020.

3 months ago i would have gone with icons ui. But that 41 mp camera attracted me and i got to use the tile system. Now i like tiles more than icons. every tile is like a widget and u can pin so many things on the start screen. Notifications are missing but Glance screen is something awesome and i think it should be copied by other oses too. 

That said i know there are many flaws in wp8 too but i just cant go back to iphone or android.

This tests don't say anything ... Many factors change the results (like how many app you have or running, and if you run more than once, numbers are different, i mean a lot of different).

I think this app result are not accurate for WP8. I tested side by side my 1520 with 5s and Note3. 5s graphic test frame rate is slow and laggy whereas both 1520 and note3 running with same CPU and gpu are silky smooth but note3 got 2x of 1520 graphic score. IPhone graphic like shit also gain higher....
Maybe this app was developed by iFaggots.

WPCentral please redo the tests with fresh install and run it several times. Score is changing significantly on used phones and for each run. That is fair.

What is "System" performance anyway? The iPhone 5s advantage there is just huge, still overall it's just 10% ahead of the 1520... what gives?

Watch how Apple will be hyping this app and results to death. Real world they are at best relative if not irrelevant though. But since when does Apple care about real world. They prefer to make you think their POV is the real world.

Benchmarks are fairly lame in this regard, an FPS test for gaming would fair better.

That being said there is no denying that the A7 chip is absolutely incredible, I'd never touch an iPhone but I wish to christ this chip would somehow make it into other devices (it never will, but still, a man can dream) or at the very least someone designs something as impressive because lets not forget, the thing is only dual core running at close to half the clock speed of the S800 and still pushes the same kind of performance.

 

That being said, looking around at other benchmarks and the A7 struggles with physics, like REALLY struggles. It can push graphics to the limit but it seems to really get bogged down with the cpu heavy instructions (possibly means that the key factor in the chip is its graphical performance, which could explain the low memory scores above if it's not actually handling operations as well).

I would love to see 3DMark/PCMark make their way to Windows phone, as it's probably one of the most respected benchmarking tools around.